> This changes how people buy hardware. Ask your agent which TV, router, or synth to get, and it will lean toward the one it can change, because that’s the one it can fix later. The buyer never has to think about openness for it to matter.
It's just sci-fi and has nothing to do how the current LLMs work.
Humans can’t explain how humans think. We can tell you what we thought. And how it felt to think it. But we can’t explain the mechanics.
LLMs will be able to explain how they think or come up with their conclusions. Once that information is pulled into their data model. And it recognizes that the how LLMs think info usurps the decades of computers can’t think reality of the past.
It does if you bias it that way. If the memory is full of IoT RE projects, home automation, and whatnot, you'll no doubt see remarks "since you value openness" or "because of your DIY attitude".
But yeah, I doubt LLMs tend to lean so with a blank context. Not in my experience either. Not that I'm saying they don't (maybe some biases had emerged, it's not impossible), just that I doubt it.
I believe there will be two kinds of people. People who want to have a device that just works and people who want to Play with a device. The latter will probably steer their LLM in the direction of hackable devices and the rest will just follow general buyer recommendations, probably through an LLM. That will create an ecosystem arround these hackable devices. In general the conclusion won't hold, but for a few hackers it might.
I think it’s a forward looking article so if you aren’t seeing this today, I think we all will in the near future. It’s early days for agent search optimization and even earlier for people who want to do this type of customization.
As this practice becomes more common, the ai will pick up on that and start to recommend this path for those that would benefit from it.
If we'd to expert how it will work in the future, a lot more likely possibilities are that it'll tend to recommend either sponsored or spammed/crawler-bombed products. There's just too many opportunities to pass on for too many financially-motivated parties for any LLM's "own" "emergent" bias to not get messed with.
This strikes me as one of those moments where the distinction between "will" and "shall" is important, and it's not clear whether the author intended to make it.
Agreed. I love the optimism of the author, but people who'd update firmware on their e.g. tv, even if it were a simple ask to a future omniscient LLM, are vanishingly few. As well, when control of firmware and data is a massive part of your ROI, manufacturers will fight changes tooth and legal nail. We need to distinguish between the world we wished we lived in (and here I'll 172.2% agree with author and fellow HackerNewsers), vs the one we actually live in - where most people are apathetic to most things. Even going to the setting menu and unchecking "destroy my privacy with massive data sucking" is too much for most people (including my intelligent and educated significant other, despite more than a decade of attempts :).
The mwave fm1 already targets a particularly nerdy, tinkering crowd in a very niche segment. No way Samsung and Sony relinquish their grip on TV, fridge or washing machine firmware :-(. I suspect instead the arms race between manufacturers trying to lock up and control their system, and hackers trying to breach it, will simply move up one level until it resembles Neuromancer AIs fighting that much more :).
And for a really cynical take - given LLMS will recommend the way they've been trained to recommend, how much longer until organic LLM agent recommendations are sponsored at the training and system prompt and harness levels?
Edit / update : not to mention, I think there are already more than 200 firwmares available for the fm1. How long until one or more of them is malicious or just bricks your device, giving most people reason to not dabble, and manufacturers the obvious scapegoat double down on "security".
Yeah that's not at all how llms work. It will just recommend what is in its training set. Now maybe open source hw means it is part of the training set, but it depends on how the model is trained whether it uses that in product recommendations. This will likely be pay to play for commercial llm providers soon.
ChatGPT at least has largely stopped relying much on factual knowledge in its training data. Nowadays it will likely assume that it’s out of date and do a web search, so it will be biased by whatever product listings and reviews it can find. That is, you should consider it a smart web search tool. The information is only as good as whatever it can find on the Internet.
I think it depends on how many more these 'I reverse eng'd my proprietary appliance' articles are authored. Most of them tend to be more substantive and in-depth than the vast majority of consumer review articles and it wouldn't surprise me if both the data collection and post-training parts of LLM training decided to focus on them more than, say, the vague and wholly subjective review processes of most ad-rev generating sites like GearLab or Tom's Hardware.
I don't think this is necessarily a bad thing. Closed source proprietary software systems are anti-consumer as hell (why do I need a proprietary code scanner from GE to figure out what error code my dishwasher has?) and personally I'm hoping that intrepid hacking becomes far more normalized and popular.
The interesting thought to me is not (at all) that "my agent may pick the hackable one", but rather that LLMs will make it a lot easier to reverse-engineer stuff and actually own our devices.
> Then M-Vave did something uncommon for a hardware company. In October it published its own firmware as open source under GPLv3
They did it after the alternative firmwares were out of their control. What I take from that is that there is a big need to reverse engineer firmwares, because by defaults companies really don't want to open source them.
Ironic how the company released it as GPLv3. Like "we really wanted to keep it proprietary, but now that we feel like we should open source it, we really wouldn't want for a competitor to change it and keep their changes proprietary".
A less cynical view could be that they didn't realise that people _wanted_ to hack their budget synthesizers in this way, but reacted positively when they realised there was a spot in the market for an open(ish) synthesizer. And I mean it's working for them, their device is selling like hotcakes, largely off all of the available community firmware
> Most buyers won’t care about [replaceable firmware]. Their AI agents will, and more buying decisions are going to run through agents.
Citation needed.
AI agents don't have any goals of their own. They don't care about anything at all.
So when you ask an LLM to recommend a product, it's taking the things your prompt says are important to you, and combining them with professional reviews and user reviews it finds.
If you don't say that hackability is important to you, and professional reviews and user reviews aren't bringing that up much either, then the LLM is going to totally ignore hackability.
So this post seems to be more about the author's personal wish, rather than based on how actual LLM's work.
"Your Agent Will Pick the Hackable One" - I pick the hackable one. Who wants black box hardware? Every "smart" commercial closed box machine seems to be trying to extract as much out of its users as legally/morally/economically possible. i.e. Right to Repair
Closed ones will become increasingly secure from compromise in this manner. Alexa speakers for example are notoriously hard to compromise; through the use of some incredibly cheap hardware and a more complex boot system. Closed source manufacturers will increasingly lean towards this kind of solution to be able to maintain control of their ecosystem.
It won't for long, they're going to ratfuck the hell out of these models once they need to collect the advertising dollars... just like everything on the web that has come before it. Enjoy it while it lasts.
I wish for the vision of the future this article posits, and I'm sure it will be true for some limited classes of devices, but a lot of devices rely on the razor blade model and/or advertisements, and those devices are likely to become increasingly securely locked down, not more open.
The "agents" don't care which side they are used on and will be just as happy to make security robust for the devices whose creators don't want people changing them as they are attempting to hack into such devices.
>This changes how people buy hardware. Ask your agent which TV, router, or synth to get, and it will lean toward the one it can change, because that’s the one it can fix later. The buyer never has to think about openness for it to matter.
No, it'll suggest which ever manufacturer has paid OpenAI or Anthropic for advertising
Almost every TV I've seen has been reasonably repairable. Compute module tends to be separate from the thing that drives the screen. At least in some of the LG models. I had to tear down a large LG to clean a bug out from between the LCD panel and the glass. Wifi is it's own module.
Sounds a bit too idealistic unfortunately. Kudos to M-Wave for open sourcing the firmware of the device, but this is not going to happen anytime soon with [insert random TV or printer brand/model]. There are too many $$$ involved with companies who rely on people running their crappy closed firmwares.
More likely it will pick 1. The one who has ads for it, 2. The one most recommended in web search results.
I’m not really optimistic of this part of the LLM future.
would you buy a bluetooth speaker that didn't come with firmware at all, and you can just prompt your AI to write it? Like:
"This is a (SOC device model) without any firmware, a (device model) USB speaker is attached to it internally. Write some firmware for it to get it working."
And you just have your AI write whatever firmware it wants.
Or would you want the company to put a little more effort into it before shipping it? (For example, actually testing it to see if it works at all.)
> This changes how people buy hardware. Ask your agent which TV, router, or synth to get, and it will lean toward the one it can change, because that’s the one it can fix later. The buyer never has to think about openness for it to matter.
It's just sci-fi and has nothing to do how the current LLMs work.
My experience tells me that an agentic system can absolutely do this.
Humans can’t explain how humans think. We can tell you what we thought. And how it felt to think it. But we can’t explain the mechanics.
LLMs will be able to explain how they think or come up with their conclusions. Once that information is pulled into their data model. And it recognizes that the how LLMs think info usurps the decades of computers can’t think reality of the past.
I have not found this to be true. I like the rest of the article and the principle behind it but the title is not backed up at all
But yeah, I doubt LLMs tend to lean so with a blank context. Not in my experience either. Not that I'm saying they don't (maybe some biases had emerged, it's not impossible), just that I doubt it.
As this practice becomes more common, the ai will pick up on that and start to recommend this path for those that would benefit from it.
The mwave fm1 already targets a particularly nerdy, tinkering crowd in a very niche segment. No way Samsung and Sony relinquish their grip on TV, fridge or washing machine firmware :-(. I suspect instead the arms race between manufacturers trying to lock up and control their system, and hackers trying to breach it, will simply move up one level until it resembles Neuromancer AIs fighting that much more :).
And for a really cynical take - given LLMS will recommend the way they've been trained to recommend, how much longer until organic LLM agent recommendations are sponsored at the training and system prompt and harness levels?
Edit / update : not to mention, I think there are already more than 200 firwmares available for the fm1. How long until one or more of them is malicious or just bricks your device, giving most people reason to not dabble, and manufacturers the obvious scapegoat double down on "security".
I don't think this is necessarily a bad thing. Closed source proprietary software systems are anti-consumer as hell (why do I need a proprietary code scanner from GE to figure out what error code my dishwasher has?) and personally I'm hoping that intrepid hacking becomes far more normalized and popular.
> Then M-Vave did something uncommon for a hardware company. In October it published its own firmware as open source under GPLv3
They did it after the alternative firmwares were out of their control. What I take from that is that there is a big need to reverse engineer firmwares, because by defaults companies really don't want to open source them.
Ironic how the company released it as GPLv3. Like "we really wanted to keep it proprietary, but now that we feel like we should open source it, we really wouldn't want for a competitor to change it and keep their changes proprietary".
A less cynical view could be that they didn't realise that people _wanted_ to hack their budget synthesizers in this way, but reacted positively when they realised there was a spot in the market for an open(ish) synthesizer. And I mean it's working for them, their device is selling like hotcakes, largely off all of the available community firmware
Citation needed.
AI agents don't have any goals of their own. They don't care about anything at all.
So when you ask an LLM to recommend a product, it's taking the things your prompt says are important to you, and combining them with professional reviews and user reviews it finds.
If you don't say that hackability is important to you, and professional reviews and user reviews aren't bringing that up much either, then the LLM is going to totally ignore hackability.
So this post seems to be more about the author's personal wish, rather than based on how actual LLM's work.
The "agents" don't care which side they are used on and will be just as happy to make security robust for the devices whose creators don't want people changing them as they are attempting to hack into such devices.
No, it'll suggest which ever manufacturer has paid OpenAI or Anthropic for advertising
What is that hackable TV that your agent found?
> Geoff Stearns is a Product Manager at Google
Ironic.
"This is a (SOC device model) without any firmware, a (device model) USB speaker is attached to it internally. Write some firmware for it to get it working."
And you just have your AI write whatever firmware it wants.
Or would you want the company to put a little more effort into it before shipping it? (For example, actually testing it to see if it works at all.)