Our developers managed to run around 750MB per website open once.
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
PSA for those who aren’t aware: Chromium/Firefox-based browsers have a Network tab in the developer tools where you can dial down your bandwidth to simulate a slower 3G or 4G connection.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
I once spent around an hour optimizing a feature because it felt slow - turns out that the slower simulated connection had just stayed enabled after a restart (can’t remember if it was just the browser or the OS, but I previously needed it and then later just forgot to turn it off). Good times, useful feature though!
I still test mine on GPRS, because my website should work fine in the Berlin U-Bahn. I also spent a lot of time working from hotels and busses with bad internet, so I care about that stuff.
Developers really ought to test such things better.
Thank you for doing this! I really mean it. We need more developers who care about keeping websites lean and fast.
There's no good reason a regular site shouldn't work on GPRS, except maybe if the main content is video.
Gonna bookmark that article for tomorrow, craptop duty is such a funny way to put it.
Similarly, a colleague I had before insisted on using a crappy screen. Helped a lot to make sure things stay visible on customers’ low contrast screens with horrible viewing angles, which are still surprisingly common.
I now wonder if it'd be a good idea to move our end to end tests to a pretty slow vm instead of beefy 8 core 32gb ram machine and check which timeouts will be triggered because our app may have been unoptimized for slower environments...
For blocking presubmit checks, getting the fastest machine you can is probably reasonable. Otherwise, the advantage of the craptop approach is that it needs basically no infra work and gives an immediate impression of the site, and not CI, being slow.
If you’re willing to build some infra, there’s probably a lot more you can do—nightly slow-hardware runs come to mind immediately, browser devtools have a convincing builtin emulation of slow connections, a page displaying a graph of test runtime over time[1] isn’t hard to set up, etc.—but I don’t really have experience with that.
I kid you not a few jobs ago I found several race conditions in my code and tests by running them at the same time as a multi threaded openssl burn test. :)
Music producers often have some shitty speakers known as grot boxes that they use to make sure their mix will sound as good as it can on consumer audio, not just on their extremely expensive studio monitors. Chromebooks are perfectly analogous. As a side note, today I learned that Grotbox is now an actual brand: https://grotbox.com
If you want to see context aware pre-fetching done right go to mcmaster.com ...
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
I used the text web (https://text.npr.org and the like) thru Lyx. Also, Usenet, Gopher, Gemini, some 16 KBPS opus streams, everything under 2.7 KBPS when my phone data plan was throttled and I was using it in tethering mode. Tons of sites did work, but Gopher://magical.fish ran really fast.
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
I'm pretty damn sure those videos were put on the page because someone in marketing wanted them. I'm pretty sure then QA complained the videos loaded too slowly, so the preloading was added. Then, the upper management responsible for the mess shrugged their shoulders and let it ship.
You're not insightful for noticing a website is dog slow or that there is a ton of data being served (almost none of which is actually the code). Please stop blaming the devs. You're laundering blame. Almost no detail of a web site or app is ever up to the devs alone.
From the perspective of the devs, they expect that the infrastructure can handle what the business wanted. If you have a problem you really should punch up, not down.
> Please stop blaming the devs. You're laundering blame. Almost no detail of a web site or app is ever up to the devs alone.
If a bridge engineer is asked to build a bridge that would collapse under its own weight, they will refuse. Why should it be different for software engineers?
It's a website and not a bridge. Based on the description given, it's not a critical website either. If it was, the requirements would have specified it must be built differently.
You're not even arguing with me BTW. You're arguing against the entire premise of running a business. Priorities are not going to necessarily be what you value most.
> If it was, the requirements would have specified it must be built differently.
I’ve seen a lot of times where “business people” ask for a feature that sounds good but isn’t technically viable for any number of reasons. The devs not doing pushback would lead to similarly non-functional/broken stuff getting shipped.
The pushback doesn’t even need to be adversarial, just do some requirements engineering, figure out what they want and go “Okay, to implement X in the best possible way, we should do Y and avoid Z because of W.”
In the bridge analogy, the people who are asking for a specific design might not know that it’d collapse under its own weight and the engineers should look for the best solution.
There are environments where devs can't do that sort of requirements engineering and those are generally pretty dysfunctional - obviously you don't need that for every feature request, but it's nice to have that ability be available when needed.
this isn't purely laundering blame. it is frustrating for the infrastructure/operations side is that the dev teams routinely kick the can down to them instead of documenting the performance/reliability weak points. in this case, when someone complains about the performance of the site, both dev and qa should have documented artifacts that explain this potential. as an infrastructure and reliability person, i am happy to support this effort with my own analysis. i am less inclined to support the dev team that just says, "hey, i delivered what they asked for, it's up to you to make it functional."
> From the perspective of the devs, they expect that the infrastructure can handle what the business wanted. If you have a problem you really should punch up, not down.
this belittles the intelligence of the dev team. they should know better. it's like validating saying "i really thought i could pour vodka in the fuel tank of this porsche and everything would function correctly. must be porsche's fault."
I agree except for your definition of "developers". I see this all the time and can't understand why the blame can't just be the business as a whole instead of singling out "developers". In fact, the only time I ever hear "developers" used that way it's a gamer without a job saying it.
The blame clearly lies with the contradictory requirements provided by the broader business too divorced from implementation details to know they're asking for something dumb. Developers do not decide those.
There’s a magic word that can be used in scenarios like this: “No.”
Failing that, interpret the requirements.
Nobody can watch a bunch of videos at once that don’t even show up until you scroll! That’s a nonsense requirement and the dev’s failure to push back or redirect in a more viable direction is a sign of their incompetence, not that of the non-technical manager that saw YouTube’s interface and assumes that that’s normal and doable.
It is! You’d have to know about lazy loading and CDNs, but neither is black magic.
It's not a sign of their incompetence, it's a sign of the realities of many corporate environments.
But hey, if you want to rail against incompetent developers who exist in a make-believe world where they hold all the power are simply too lazy and incompetent to 'do the right thing' then go ahead!
There’s nothing “make believe” here, incompetent devs, and devs (regardless of competence) who don’t push back against silly requirements _absolutely_ exist.
Stop making excuses and start taking ownership and responsibility of your craft.
I work in huge government departments, large financial orgs, and other "enterprise" places that are the poster child for the "realities of corporate environments".
Automatically saying "yes" to everything makes you a useless meat robot.
If you do everything that the customer asks, without push back, negotiation, or at least a deeper understanding, then you will produce broken garbage.
I see this all the time: "The customer asked for X, so I pressed the button!" is the cry of the incompetent junior tech that will never be promoted.
Nobody wants a uselessly slow website. Nobody wants to piss of their customers. Nobody wants angry rants about their online presence to make headline news.
What the customer wanted was multi media content. That's fine. The technical specifics of how that is presented is up to the engineering team to decide. You're not advisors! You own the technical decision making, so act like it.
If you make the decision to shove nearly a gigabyte down the wire to show the landing page, then that's on you. The manager asking for "video clips" or whatever as the feature probably doesn't even know the difference between megabyte and gigabyte! They shouldn't have to in the same way that I shouldn't have to know about my state's electrical wiring standards if I get a sparky out to add a porch light. If my house burns down, that's the electrician's fault, not mine as the customer!
Similarly, if someone asks for lights inside their pool, an electrician that strings ordinary mains cabling through the water should be jailed for criminal negligence. Obviously, only special low-voltage lighting can be used in water, especially near people. Duh.
Act like an electrician, not like a bored shopkeer who's memorised the line "the customer is always right" without realising that the full quote ends in "... in matters of taste."
The devs are the subject matter experts. Does marketing understand the consequences of preloading all those videos? Does upper management? Unlikely. It’s the experts’ job to educate them. That’s part of the job as much as writing code is.
From the perspective of the devs, they have a responsibility for saying something literally wont fly anywhere, ever, saying the business is responsible for every bad decision is a complete abrogation of your responsibilities.
Why don't you tell your boss or team something like that and see how well that flies.
The responsibility of the devs is to deliver what was asked. They can and probably do make notes of the results. So does QA. So do the other stakeholders. On their respective teams they get the same BS from everyone who isn't pleased with the outcome.
Ultimately things are on a deadline and the devs must meet requirements where the priority is not performance. It says nothing about their ability to write performant code. It says nothing about whether that performant code is even possible in a browser while meeting the approval of the dozens of people with their own agendas. It says everything about where you work.
Maybe everyone’s got a different situation, but when a different department tried to put ActiveX avatars all over their site, though it offended me from a UX perspective, I was able to get higher ups to reject it by pointing out that it would shut out 20% of their customers.
We always have discussions here about how you have to learn to talk to communicate your value to clients in a language they understand. Same goes for internal communications.
> The responsibility of the devs is to deliver what was asked.
Software development isn't factory work. And factory workers are expected to notice problems and escalate them.
Anyway, they're paying me far too much to have me turn off my brain and just check the boxes they want checked in all situations. Sometimes, checking boxes because they need to be checked is the thing to do, but usually it's not.
I didn't say anything about their development abilities, what I am pointing to is their professional responsibility. If a doctor is asked by a client to cut off their arm and they say no, and the client fires them, did the doctor err? (No) This doesn't comment on their ability to do surgery.
So just to check, instead of doing something you were told to, that you know is a stupid idea (after telling all concerned it's a dumb idea and being told to go ahead anyway, eg adding a crapton of video to a page), you would just resign, to protect your personal integrity?
I just loaded the nytimes.com page as an experiment. The volume of tracking pixels and other ad non-sense is truly horrifying.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
It would be less hypocritical if that critique of the situation wasn't posted on a website that itself loads unnecessary 3rd party resources (e.g. cloudflare insights).
Luckily I use a proper content blocker (uBlock Origin in hard mode).
I was really surprised when I went to book a flight on Frontier (don't judge me!) and a request from analytics.tiktok.com loaded. I have a lot of discomfort about that. Bloat and surveillance go hand in hand.
These days the NYT is in a race to the bottom. I no longer even bother to bypass ads let alone read the news stories because of its page bloat and other annoyances. It's just not worth the effort.
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
In the past some site had light versions, but I haven’t come across one in over 10 years
Makes me wonder if this isn’t just some rogue employee maintaining this without anyone else realizing it
It’s the light version, but ironically I would happily pay these ad networks a monthly $20 to just serve these lite pages and not track me. They don’t make anywhere close to that from me in a year
Sadly, here’s how it would go: they’d do it, it be successful, they’d ipo, after a few years they’d need growth, they’d introduce a new tier with ads, and eventually you’d somehow wind up watching ads again
> Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
They know this. They also know that web surfers like you would never actually buy a subscription and you have an ad blocker running to deny any revenue generation opportunities.
Visitors like you are a tiny minority who were never going to contribute revenue anyway. You’re doing them a very tiny favor by staying away instead of incrementally increasing their hosting bills.
"Why would you feel guilty for not visiting a site you’re not paying for and where you’re blocking ads?"
This isn't a simple as it sounds, in fact it's rather complicated (far too involved to cover in depth here).
In short, ethics are involved (and believe it or not I actually possess some)!
In the hayday of newsprint people actually bought newspapers at a cheap affordable price and the bulk of their production was paid for by advertisements. We readers mostly paid for what we read, newspapers were profitable and much journalism was of fair to good quality. Back then, I had no qualms about forking out a few cents for a copy of the NYT.
Come the internet the paradigm changed and we all know what happened next. In fact, I feel sorry about the demise of newsprint because what's replaced it is of significantly lesser value.
In principle I've no objection to paying for news but I will not do so for junk and ads that I cannot avoid (with magazines and newspapers ads are far less intrusive).
So what's the solution? It's difficult but I reckon there are a few worth considering. For example, I mentioned some while ago on HN that making micro payments to websites ought to be MUCH easier than it is now (this would apply to all websites and would also be a huge boon for open source developers).
What I had in mind was an anonymous "credit" card system with no strings attached. Go to your local supermarket, kiosk or whatever and purchase a scratchy card with a unique number to say the value of $50 for cash and use that card to make very small payments to websites. Just enter the card's number and the transaction is done (only enter one's details if purchasing something that has to be delivered).
That way both the card and user remain anonymous if the user wishes, also one's privacy is preserved, etc. It could be implemented by blockchain or such.
The technical issues are simple but
problems are obvious—and they're all political. Governments would go berserk and cry money laundering, tax evasion, criminal activity, etc., and the middlemen such as Master and Visa cards would scream to high heaven that their monopolies were being undercut.
In short, my proposal is essentially parallels what now exits with cash—I go to a supermarket and pay cash for groceries, the store doesn't need to know who I am. It ought to be no big deal but it isn't.
It seems to me a very simple micro payments system without name, rank and serial number attached would solve many of the internet payment problems.
Sure, there'll always be hardline scavengers and scrapers but many people would be only too happy to pay a little amount for a service they wanted, especially so when they knew the money was going into producing better products.
For example, I'd dearly love to be able to say purchase a copy of LibreOffice for $10 - $20 and know there was enough money in the organisation to develop the product to be fully on par with MSO.
Trouble is when buying stuff on the internet there's a minimum barrier to overcome and it's too high for most people when it comes to making micro payments (especially when the numbers could run into the hundreds per week).
I cannot understand why those who'd benefit from such a scheme haven't at least attempted to push the matter.
> Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
Seems like a gross overestimation of how much facility people have with computers but they don't want random article readers anyway; they want subscribers who use the app or whatever.
Do you think youtube will continue to make it possible to use alternate clients, or eventually go the way of e.g. Netflix with DRM so you're forced to use their client and watch ads?
If Google were just starting YouTube today then DRM would likely be enforced through a dedicated app. The trouble for Google is that millions watch YouTube through web browsers many of whom aren't even using a Google account let alone even being subscribers to a particular YouTube page. Viewership would drop dramatically.
Only several days ago I watched the presenter of RobWords whinging about wanting more subscribers and stating that many more people just watch his presentations than watch and also subscribe.
The other problem YouTube has is that unlike Netflix et al with high ranking commercial content are the millions of small presenters who do not use advertising and or just want to tell the world at large their particular stories. Enforced DRM would altogether ruin that ecosystem.
Big tech will slowly enforce "secure browsing" and "secure OS" in a way that will make it impossible to browse the web without a signed executable approved by them. DRM is just a temporary stopgap.
It doesn't have to be that way, you can only push people so far before they riot. History has thousands of instances and many have been very ugly, 1789 and 1917 for instance.
What does playing fair mean in this context? It would be one thing if you were a paid subscriber complaining that even paying sucks so you left, but it sounds like you’re not.
It is strange to hear these threats about avoiding websites from people who are not subscribers and also definitely using an ad blocker.
News sites aren’t publishing their content for the warm fuzzy feeling of seeing their visitor count go up. They’re running businesses. If you’re dead set on not paying and not seeing ads, it’s actually better for them that you don’t visit the site at all.
'Running a business' is not carte blanche to do whatever you like to get money, and it does not silence valid criticism. Businesses still exist in society and have to act accordingly. A primary mechanism that society has to enforce rules is criticism and shame.
I'd like to answer that in detail but it's impractical to do so here as it'd take pages. As a starter though begin with them not violating users' privacy.
Another quick point: my observation is that the worse the ad problem the lower quality the content is. Cory Doctorow's "enshitification" encapsulates the problems in a nutshell.
You're right, it means nothing. But it cuts two ways. These sites are sending me bytes and I choose which bytes I visualize (via an ad blocker). Any expectation the website has about how I consume the content has no meaning and it's entirely their problem.
> Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
No.
"savvy" web surfers are a rounding error in global audience terms. Vast majorities of web users, whether paying subscribers to a site like NYT or not, have no idea what a megabyte is, nor what javascript is, nor why they might want to care about either. The only consideration is whether the site has content they want to consume and whether or not it loads. It's true that a double digit % are using ad blockers, but they aren't doing this out of deep concerns about Javascript complexity.
Do what you have to do, but no one at the NYT is losing any sleep over people like us.
"…but no one at the NYT is losing any sleep over people like us."
Likely not, but they are over their lost revenues. The profitability of newspapers and magazines has been slashed to ribbons over the past couple of decades and internet revenues hardly nudge the graphs.
Internet beneficiaries are all new players, Google et al.
I also use and like the comparison in units of Windows 95 installs (~40MB), which is also rather ironic in that Win95 was widely considered bloated when it was released.
While this article focuses on ads, it's worth noting that sites have had ads for a long time, but it's their obnoxiousness and resource usage that's increased wildly over time. I wouldn't mind small sponsored links and (non-animated!) banners, but the moment I enable JS to read an article and it results in a flurry of shit flying all over the page and trying to get my attention, I leave promptly.
I would love for someone more knowledgeable in this space than I to chime in on the economics of this industry
Are the few cents you get from antagonizing users really worth it?
I suspect the answer is simple and that most users don’t give a shit
I think it has to do a lot with when you came of age - I’m in my late 30s, I got my first tech job at 14 as a sys admin for a large school district, and every single developer, admin, etc that I knew was already going on about the free internet. As a result, I’ve never had a tolerance for anything but the most reasonable advertisements
I think that ideology is necessary to care enough and be motivated enough to really get rid of ads, how fucking awful the websites are alone should be enough but for most people it isn’t
Modern web dev is ridiculous. Most websites are an ad ridden tracking hellacape. Seeing sites like hn where lines of js are taken seriously is a godsend. Make the web less bloated.
What really blows my mind is how unusable it is. How often do you visit a web site and there’s so much crap overlaid on the page that you can barely see the actual content? Surely that can’t be good for their ability to make money, yet they persist.
This is just the top of the iceberg. Don't get me started on airlines websites (looking at you Air Canada), where the product owner, designers, developers are not able to get a simple workflow straight without loading Mb of useless javascript and interrupt the user journey multiple times.
Give me back the command line terminal like Amadeus, that would be perfect.
How can we go back to a Web where websites are designed to be used by the user and not for the shareholders?
How can we go back to a Web where websites are designed to be used by the user and not for the shareholders?
Loudly oppose the trendchasing devs who have been brainwashed into the "newer is better" mindset by Big Tech. I'm sure the shareholders would want to reduce the amount they spend on server/bandwidth costs and doing "development and maintenance" too.
Simple HTML forms can already make for a very usable and cheap site, yet a whole generation of developers have been fed propaganda about how they need to use JS for everything.
You can't beat China Southern . They have the most dog shit website I've ever seen. The flight was fine but I gave up doing online check in after 3 attempts. Never mind the bloat:
- required text fields with wrong or missing labels. One field was labeled "ticket no.". It kept getting rejected. I randomly tried passport number instead. It worked.
- sometimes fields only have a placeholder that you can't fully read because the field has not enough width ("Please enter the correct...") and the placeholder disappears once you start typing.
- date picker is randomly in Chinese
- makes you go through multi step seat selection process only to tell you at the end that seat selection is not possible anymore.
- signed up with email; logged out and went back to the SAME login page; now sign up via phone number is required!?
My family's first broadband internet connection, circa 2005, came with a monthly data quota of 400 MB.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
I remember getting punishment from parents for downloading 120MB World of Tanks update over metered home internet. Our monthly quota was 250MB. It was not that long ago, 2010.
It's really hard to consider any kind of web dev as "engineering." Outcomes like this show that they don't have any particular care for constraints. It's throw-spaghetti-at-the-wall YOLO programming.
There are plenty of web devs who care about performance and engineering quality. But caring about such things when you work on something like a news site is impossible: These sites make their money through user tracking, and it's literally your job to stuff in as many 3rd-party trackers as management tells you to. Any dev who says no on the basis that it'll slow the site down will get fired as quickly as a chef who get a shift job in McDonalds and tries to argue for better cuisine.
Allowing scripting on websites (in the mid-90s) was a completely wrong decision. And an outrage. Programs are downloaded to my computer and executed without me being able to review them first—or rely on audits by people I trust. That’s completely unacceptable; it’s fundamentally flawed.
Of course, you disable scripts on websites. But there are sites that are so broken that they no longer work properly, since the developers are apparently so confused that they assume people only view their pages with JavaScript enabled.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.
There is obviously huge demand for scripting on websites. There is no one authority on what gets allowed on the web, if the existing orgs didn't implement it, someone else would have and users would have moved over when they saw they could access new more capable, interactive pages.
The 49MB webpage just shows what our priorities are. It shows the target audience has fast internet that can load this without issues. On my average home connection in Australia, I can download a 49MB page in 0.3 seconds. We spend time optimising for what matters to the end user.
Stepping back, it's pretty ridiculous that I need to download executable code, often bloated, solely to view read-only content. Just render the thing on the backend and send it to the client.
This is why people continue to lament Google Reader (and RSS in general): it was a way to read content on your own terms, without getting hijacked by ads.
What on earth do you have to rely on alphabet, an ad company, to read rss for? there are many other options, that are not made by an ad company.
Google Reader was never the answer. It's such a shame that people even here don't realize that relying on Google for that had interests at odds - and you weren't part of the equation at all.
Well, except for your data. You didn't give them enough data. So they shut down shop. Gmail though, ammirite? :D
Yeah I wonder why gmail was not one of the shut down products /s
It's almost criminal that the article does not mention network-wide DNS blocklists as an obvious solution to this problem. I stop nearly 100% of ads in their tracks using the Hagezi ultimate list, and run uBlock on desktop for cosmetic filtering and YouTube.
I should really run some to tests to figure out how much lighter the load on my link is thanks to the filter.
I also manually added some additional domains (mostly fonts by Google and Adobe) to further reduce load and improve privacy.
This rubbish also exists disproportionately for recipe pages/cooking websites as well.
You have 20 ads scattered around, an autoplaying video of some random recipe/ad, 2-3 popups to subscribe, buy some affiliated product and then the author's life story and then a story ABOUT the recipe before I am able to see the detailed recipe in the proper format.
It's second nature to open all these websites in reader mode for me atp.
I remember in 2008, when Wizards of the Coast re-launched the official Dungeons & Dragons website to coincide with the announcement of the fourth edition rules. The site was something in the region of 4 MB, plus a 20 MB embedded video file. A huge number of people were refreshing the site to see what the announcement was, and it was completely slammed. Nobody could watch the trailer until they uploaded it to YouTube later.
4 MB was an absurd size for a website in 2008. It's still an absurd size for a website.
Even enterprise COTS products can have some of these issues. We have an on-premise Atlassian suite, and Jira pages sometimes have upwards of 30MB total payloads for loading a simple user story page — and keep in mind there is no ad-tech or other nonsense going on here, it’s just pure page content.
This site more or less practices what it preaches. `newsbanner.webp` is 87.1KB (downloaded and saved; the Network tab in Firefox may report a few times that and I don't know why); the total image size is less than a meg and then there's just 65.6KB of HTML and 15.5 of CSS.
And it works without JavaScript... but there does appear to be some tracking stuff. A deferred call out to Cloudflare, a hit counter I think? and some inline stuff at the bottom that defers some local CDN thing the old-fashioned way. Noscript catches all of this and I didn't feel like allowing it in order to weigh it.
Only major media can get away with this kind of bloat. For the normal website, Google would never include you in the SERPs even if your page is a fraction of that size.
I think it's a GOOD thing, actually. Because all these publications a dying anyway. And even if your filter out all the ad and surveillance trash, you are left with trash propaganda and brain rot content. Like why even make the effort of filtering out the actual text from some "journalist" from these propaganda outlets. It's not even worth it.
If people tune out only because how horrible the sites are, good.
rule #1 is to always give your js devs only core 2 quad cpus + 16GB of RAM
they won't be able to complain about low memory but their experience will be terrible every time they try to shove something horrible into the codebase
The sad thing is, this is already a paywalled site.
I’m afraid someone who wants to support professional journalism and agrees to pay ~$300/yr for an NYT subscription still gets most (all?) of this nonsense?
I worked at big newspapers as a software engineer. Please do not blame the engineers for this mess. As the article says news is in a predicament because of the ads business model. Subscriptions alone usually cannot cover all costs and ads will invariably make their way in.
For every 1 engineer it seems like there are 5 PMs who need to improve KPIs somehow and thus decide auto playing video will improve metrics. It does. It also makes people hate using your website.
I would constantly try to push back against the bullshit they'd put on the page but no one really cares what a random engineer thinks.
I don't think there's any real way to solve this unless we either get less intrusive ad tech or news gets a better business model. Many sites don't even try with new business models, like local classifieds or local job boards. And good luck getting PMs to listen to an engineer talking about these things.
Maybe I'm just getting old, but I've gotten tired of these "Journalists shouldn't try to make their living by finding profitable ads, they should just put in ads that look pretty but pay almost nothing and supplement their income by working at McDonalds" takes.
I'm pretty sure people would read more and click on more ads if they didn't have to endure waiting for 49 MB of crap and then navigating a pop-up obstacle course for each article.
This is what killed my willingness to subscribe to most outlets. If I'm paying, I expect the page to load in under a second with zero tracking. Instead you get the same bloated experience minus a banner ad or two.
This argument is valid if journalism was actually journalism instead of just ripping off trending stories from HN and Reddit and rehashing it with sloppy AI and calling it a day and putting in 4 lines of text buried inside 400 ads.
I don't like the state of journalism either but you realize this is a vicious cycle, no? People not paying for news (by buying newspaper, or more importantly paying for classified ads) leading to low quality online reporting leading to people not wanting to pay for online news.
I never understand this type of comment. People don't pay for news so newspapers (which by the way have pay walls) are forced to degrade their service. It seems strange to me. If I have a restaurant and people don't want to pay for my food, making even worse food with worse service doesn't seem a good solution. If I write books and people don't buy them, writing worse books doesn't make my sales better. Why journalists are different? They sell a service for money like all the others, but for some reason they have a special status and it's totally understandable that they respond to bad sales with a worse product. And actually, somehow it's our fault as customers. For some reason we should keep buying newspapers even if we don't think it's worth to save them from themselves.
Using your analogy, if every restaurant in town had a problem where most people wanted to come in and get food for free (and it was an expectation in the industry) and people refused to go in and pay, everyone would be upset they could no longer go out to eat when there were none left. If nobody is interested in paying for their meal, you can't be shocked the ingredient and chef quality drops in turn.
They have put in ticket with ops that the server is slow and could we look at it. So we looked. Every single video on a page with long video list pre-loaded a part of it. The single reason the site didn't ran like shit for them is coz office had direct fiber to out datacenter few blocks away.
We really shouldn't allow web developers more than 128kbit of connection speed, anything more and they just make nonsense out of it.
Combined with CPU throttling, it's a decent sanity check to see how well your site will perform on more modest setups.
Developers really ought to test such things better.
[1] https://css-tricks.com/test-your-product-on-a-crappy-laptop/
Similarly, a colleague I had before insisted on using a crappy screen. Helped a lot to make sure things stay visible on customers’ low contrast screens with horrible viewing angles, which are still surprisingly common.
If you’re willing to build some infra, there’s probably a lot more you can do—nightly slow-hardware runs come to mind immediately, browser devtools have a convincing builtin emulation of slow connections, a page displaying a graph of test runtime over time[1] isn’t hard to set up, etc.—but I don’t really have experience with that.
[1] See e.g. https://arewefastyet.com/win11/benchmarks/overview?numDays=3....
Even if you don’t fix them, knowing where the weak points are is valuable for when they do snap in production.
There are good reasons to have a small cheap development staging server, as the rate-limited connection implicitly trains people what not to include. =3
Bitlbee saved (and still saves) my ass with tons of the protocols available via IRC using nearly nil data to connect. Also you can connect with any IRC client since early 90's.
Not just web developers. Electron lovers should be trottled with 2GB of RAM machines and some older Celeron/Core Duo machine with a GL 2.1 compatible video card. It it desktop 'app' smooth on that machine, your project it's ready.
You're not insightful for noticing a website is dog slow or that there is a ton of data being served (almost none of which is actually the code). Please stop blaming the devs. You're laundering blame. Almost no detail of a web site or app is ever up to the devs alone.
From the perspective of the devs, they expect that the infrastructure can handle what the business wanted. If you have a problem you really should punch up, not down.
If a bridge engineer is asked to build a bridge that would collapse under its own weight, they will refuse. Why should it be different for software engineers?
You're not even arguing with me BTW. You're arguing against the entire premise of running a business. Priorities are not going to necessarily be what you value most.
I’ve seen a lot of times where “business people” ask for a feature that sounds good but isn’t technically viable for any number of reasons. The devs not doing pushback would lead to similarly non-functional/broken stuff getting shipped.
The pushback doesn’t even need to be adversarial, just do some requirements engineering, figure out what they want and go “Okay, to implement X in the best possible way, we should do Y and avoid Z because of W.”
In the bridge analogy, the people who are asking for a specific design might not know that it’d collapse under its own weight and the engineers should look for the best solution.
There are environments where devs can't do that sort of requirements engineering and those are generally pretty dysfunctional - obviously you don't need that for every feature request, but it's nice to have that ability be available when needed.
> From the perspective of the devs, they expect that the infrastructure can handle what the business wanted. If you have a problem you really should punch up, not down.
this belittles the intelligence of the dev team. they should know better. it's like validating saying "i really thought i could pour vodka in the fuel tank of this porsche and everything would function correctly. must be porsche's fault."
The blame clearly lies with the contradictory requirements provided by the broader business too divorced from implementation details to know they're asking for something dumb. Developers do not decide those.
Failing that, interpret the requirements.
Nobody can watch a bunch of videos at once that don’t even show up until you scroll! That’s a nonsense requirement and the dev’s failure to push back or redirect in a more viable direction is a sign of their incompetence, not that of the non-technical manager that saw YouTube’s interface and assumes that that’s normal and doable.
It is! You’d have to know about lazy loading and CDNs, but neither is black magic.
But hey, if you want to rail against incompetent developers who exist in a make-believe world where they hold all the power are simply too lazy and incompetent to 'do the right thing' then go ahead!
Stop making excuses and start taking ownership and responsibility of your craft.
I work in huge government departments, large financial orgs, and other "enterprise" places that are the poster child for the "realities of corporate environments".
Automatically saying "yes" to everything makes you a useless meat robot.
If you do everything that the customer asks, without push back, negotiation, or at least a deeper understanding, then you will produce broken garbage.
I see this all the time: "The customer asked for X, so I pressed the button!" is the cry of the incompetent junior tech that will never be promoted.
Nobody wants a uselessly slow website. Nobody wants to piss of their customers. Nobody wants angry rants about their online presence to make headline news.
What the customer wanted was multi media content. That's fine. The technical specifics of how that is presented is up to the engineering team to decide. You're not advisors! You own the technical decision making, so act like it.
If you make the decision to shove nearly a gigabyte down the wire to show the landing page, then that's on you. The manager asking for "video clips" or whatever as the feature probably doesn't even know the difference between megabyte and gigabyte! They shouldn't have to in the same way that I shouldn't have to know about my state's electrical wiring standards if I get a sparky out to add a porch light. If my house burns down, that's the electrician's fault, not mine as the customer!
Similarly, if someone asks for lights inside their pool, an electrician that strings ordinary mains cabling through the water should be jailed for criminal negligence. Obviously, only special low-voltage lighting can be used in water, especially near people. Duh.
Act like an electrician, not like a bored shopkeer who's memorised the line "the customer is always right" without realising that the full quote ends in "... in matters of taste."
https://en.wikipedia.org/wiki/Conway's_law
Have a wonderful day =3
The responsibility of the devs is to deliver what was asked. They can and probably do make notes of the results. So does QA. So do the other stakeholders. On their respective teams they get the same BS from everyone who isn't pleased with the outcome.
Ultimately things are on a deadline and the devs must meet requirements where the priority is not performance. It says nothing about their ability to write performant code. It says nothing about whether that performant code is even possible in a browser while meeting the approval of the dozens of people with their own agendas. It says everything about where you work.
We always have discussions here about how you have to learn to talk to communicate your value to clients in a language they understand. Same goes for internal communications.
Software development isn't factory work. And factory workers are expected to notice problems and escalate them.
Anyway, they're paying me far too much to have me turn off my brain and just check the boxes they want checked in all situations. Sometimes, checking boxes because they need to be checked is the thing to do, but usually it's not.
But at least in terms of the headline metric of bandwidth, it's somewhat less horrifying. With my ad-blocker off, Firefox showed 44.47mb transferred. Of that 36.30mb was mp4 videos. These videos were journalistic in nature (they were not ads).
So, yes in general, this is like the Hindenburg of web pages. But I still think it's worth noting that 80% of that headline bandwidth is videos, which is just part of the site's content. One could argue that it is too video heavy, but that's an editorial issue, not an engineering issue.
Luckily I use a proper content blocker (uBlock Origin in hard mode).
Surely news outlets like the NYT must realize that savvy web surfers like yours truly when encountering "difficult" news sites—those behind firewalls and or with megabytes of JavaScript bloat—will just go elsewhere or load pages without JavaScript.
We'll simply cut the headlines from the offending website and past it into a search engine and find another site with the same or similar info but with easier access.
I no longer think about it as by now my actions are automatic. Rarely do I find an important story that's just limited to only one website, generally dozens have the story and because of syndication the alternative site one selects even has identical text and images.
My default browsing is with JavaScript defaulted to "off" and it's rare that I have to enable it (which I can do with just one click).
I never see Ads on my Android phone or PC and that includes YouTube. Disabling JavaScript on webpages nukes just about all ads, they just vanish, any that escape through are then trapped by other means. In ahort, ads are optional. (YouTube doesn't work sans JS, so just use NewPipe or PipePipe to bypass ads.)
Disabling JavaScript also makes pages blindingly fast as all that unnecessary crap isn't loaded. Also, sans JS it's much harder for websites to violate one's privacy and sell one's data.
Do I feel guilty about skimming off info in this manner? No, not the slightest bit. If these sites played fair then it'd be a different matter but they don't. As they act like sleazebags they deserve to be treated as such.
In the past some site had light versions, but I haven’t come across one in over 10 years
Makes me wonder if this isn’t just some rogue employee maintaining this without anyone else realizing it
It’s the light version, but ironically I would happily pay these ad networks a monthly $20 to just serve these lite pages and not track me. They don’t make anywhere close to that from me in a year
Sadly, here’s how it would go: they’d do it, it be successful, they’d ipo, after a few years they’d need growth, they’d introduce a new tier with ads, and eventually you’d somehow wind up watching ads again
They know this. They also know that web surfers like you would never actually buy a subscription and you have an ad blocker running to deny any revenue generation opportunities.
Visitors like you are a tiny minority who were never going to contribute revenue anyway. You’re doing them a very tiny favor by staying away instead of incrementally increasing their hosting bills.
I subscribe, and yet they still bombard me with ads. Fuck that. One reason I don’t use apps is that I can’t block ads.
For a while it looked like companies were going to offer a good product at a fair price. I started getting a few subscriptions to various services.
Then all of those services got enshitefied. I got ads in paid accounts, slow loads, obvious data mining, etc.
Paying for services now often offers a degraded experience relative to less legitimate methods of acces.
This isn't a simple as it sounds, in fact it's rather complicated (far too involved to cover in depth here).
In short, ethics are involved (and believe it or not I actually possess some)!
In the hayday of newsprint people actually bought newspapers at a cheap affordable price and the bulk of their production was paid for by advertisements. We readers mostly paid for what we read, newspapers were profitable and much journalism was of fair to good quality. Back then, I had no qualms about forking out a few cents for a copy of the NYT.
Come the internet the paradigm changed and we all know what happened next. In fact, I feel sorry about the demise of newsprint because what's replaced it is of significantly lesser value.
In principle I've no objection to paying for news but I will not do so for junk and ads that I cannot avoid (with magazines and newspapers ads are far less intrusive).
So what's the solution? It's difficult but I reckon there are a few worth considering. For example, I mentioned some while ago on HN that making micro payments to websites ought to be MUCH easier than it is now (this would apply to all websites and would also be a huge boon for open source developers).
What I had in mind was an anonymous "credit" card system with no strings attached. Go to your local supermarket, kiosk or whatever and purchase a scratchy card with a unique number to say the value of $50 for cash and use that card to make very small payments to websites. Just enter the card's number and the transaction is done (only enter one's details if purchasing something that has to be delivered).
That way both the card and user remain anonymous if the user wishes, also one's privacy is preserved, etc. It could be implemented by blockchain or such.
The technical issues are simple but problems are obvious—and they're all political. Governments would go berserk and cry money laundering, tax evasion, criminal activity, etc., and the middlemen such as Master and Visa cards would scream to high heaven that their monopolies were being undercut.
In short, my proposal is essentially parallels what now exits with cash—I go to a supermarket and pay cash for groceries, the store doesn't need to know who I am. It ought to be no big deal but it isn't.
It seems to me a very simple micro payments system without name, rank and serial number attached would solve many of the internet payment problems.
Sure, there'll always be hardline scavengers and scrapers but many people would be only too happy to pay a little amount for a service they wanted, especially so when they knew the money was going into producing better products.
For example, I'd dearly love to be able to say purchase a copy of LibreOffice for $10 - $20 and know there was enough money in the organisation to develop the product to be fully on par with MSO.
Trouble is when buying stuff on the internet there's a minimum barrier to overcome and it's too high for most people when it comes to making micro payments (especially when the numbers could run into the hundreds per week).
I cannot understand why those who'd benefit from such a scheme haven't at least attempted to push the matter.
Oh, and that's just one aspect of the problem.
Seems like a gross overestimation of how much facility people have with computers but they don't want random article readers anyway; they want subscribers who use the app or whatever.
Only several days ago I watched the presenter of RobWords whinging about wanting more subscribers and stating that many more people just watch his presentations than watch and also subscribe.
The other problem YouTube has is that unlike Netflix et al with high ranking commercial content are the millions of small presenters who do not use advertising and or just want to tell the world at large their particular stories. Enforced DRM would altogether ruin that ecosystem.
News sites aren’t publishing their content for the warm fuzzy feeling of seeing their visitor count go up. They’re running businesses. If you’re dead set on not paying and not seeing ads, it’s actually better for them that you don’t visit the site at all.
Another quick point: my observation is that the worse the ad problem the lower quality the content is. Cory Doctorow's "enshitification" encapsulates the problems in a nutshell.
No.
"savvy" web surfers are a rounding error in global audience terms. Vast majorities of web users, whether paying subscribers to a site like NYT or not, have no idea what a megabyte is, nor what javascript is, nor why they might want to care about either. The only consideration is whether the site has content they want to consume and whether or not it loads. It's true that a double digit % are using ad blockers, but they aren't doing this out of deep concerns about Javascript complexity.
Do what you have to do, but no one at the NYT is losing any sleep over people like us.
Likely not, but they are over their lost revenues. The profitability of newspapers and magazines has been slashed to ribbons over the past couple of decades and internet revenues hardly nudge the graphs.
Internet beneficiaries are all new players, Google et al.
While this article focuses on ads, it's worth noting that sites have had ads for a long time, but it's their obnoxiousness and resource usage that's increased wildly over time. I wouldn't mind small sponsored links and (non-animated!) banners, but the moment I enable JS to read an article and it results in a flurry of shit flying all over the page and trying to get my attention, I leave promptly.
Are the few cents you get from antagonizing users really worth it?
I suspect the answer is simple and that most users don’t give a shit
I think it has to do a lot with when you came of age - I’m in my late 30s, I got my first tech job at 14 as a sys admin for a large school district, and every single developer, admin, etc that I knew was already going on about the free internet. As a result, I’ve never had a tolerance for anything but the most reasonable advertisements
I think that ideology is necessary to care enough and be motivated enough to really get rid of ads, how fucking awful the websites are alone should be enough but for most people it isn’t
How can we go back to a Web where websites are designed to be used by the user and not for the shareholders?
Loudly oppose the trendchasing devs who have been brainwashed into the "newer is better" mindset by Big Tech. I'm sure the shareholders would want to reduce the amount they spend on server/bandwidth costs and doing "development and maintenance" too.
Simple HTML forms can already make for a very usable and cheap site, yet a whole generation of developers have been fed propaganda about how they need to use JS for everything.
You can't beat China Southern . They have the most dog shit website I've ever seen. The flight was fine but I gave up doing online check in after 3 attempts. Never mind the bloat:
- required text fields with wrong or missing labels. One field was labeled "ticket no.". It kept getting rejected. I randomly tried passport number instead. It worked.
- sometimes fields only have a placeholder that you can't fully read because the field has not enough width ("Please enter the correct...") and the placeholder disappears once you start typing.
- date picker is randomly in Chinese
- makes you go through multi step seat selection process only to tell you at the end that seat selection is not possible anymore.
- signed up with email; logged out and went back to the SAME login page; now sign up via phone number is required!?
Or for developers to pad their CV.
The fundamental problem of journalism is that the economics no longer works out. Historically, the price of a copy of a newspaper barely covered the cost of printing; the rest of the cost was covered by advertising. And there was an awful lot of advertising: everything was advertised in newspapers. Facebook Marketplace and Craigslist were a section of the newspaper, as was whichever website you check for used cars or real estate listings. Journalism had to be subsidised by advertising, because most people aren't actually that interested in the news to pay the full cost of quality reporting; nowadays, the only newspapers that are thriving are those that aggressively target those who have an immediate financial interest in knowing what's going on: the Financial Times, Bloomberg, and so on.
The fact is that for most people, the news was interesting because it was new every day. Now that there is a more compelling flood of entertainment in television and the internet, news reporting is becoming a niche product.
The lengths that news websites are going to to extract data from their readers to sell to data brokers is just a last-ditch attempt to remain profitable.
It would have been so much better if we had simply decided back in the ’90s that executable programs and HTML don’t belong together. The world would be so much better today.
The 49MB webpage just shows what our priorities are. It shows the target audience has fast internet that can load this without issues. On my average home connection in Australia, I can download a 49MB page in 0.3 seconds. We spend time optimising for what matters to the end user.
I didn’t bother validating this, but I’m sure they wouldn’t lie or misinterpret!!
Hacker News still has a feed (that's the only reason why I saw this thread). And Reddit gives you a feed for your subscribed subreddits.
Google Reader was never the answer. It's such a shame that people even here don't realize that relying on Google for that had interests at odds - and you weren't part of the equation at all.
Well, except for your data. You didn't give them enough data. So they shut down shop. Gmail though, ammirite? :D
Yeah I wonder why gmail was not one of the shut down products /s
I should really run some to tests to figure out how much lighter the load on my link is thanks to the filter.
I also manually added some additional domains (mostly fonts by Google and Adobe) to further reduce load and improve privacy.
Not done any rigorous tests but my experience has been rhat it can be lower than a tenth.
You have 20 ads scattered around, an autoplaying video of some random recipe/ad, 2-3 popups to subscribe, buy some affiliated product and then the author's life story and then a story ABOUT the recipe before I am able to see the detailed recipe in the proper format.
It's second nature to open all these websites in reader mode for me atp.
4 MB was an absurd size for a website in 2008. It's still an absurd size for a website.
The answer is really simple and follows on from this article; the purpose of the app is even more privacy violation and tracking.
And it works without JavaScript... but there does appear to be some tracking stuff. A deferred call out to Cloudflare, a hit counter I think? and some inline stuff at the bottom that defers some local CDN thing the old-fashioned way. Noscript catches all of this and I didn't feel like allowing it in order to weigh it.
If people tune out only because how horrible the sites are, good.
Want to bet 100 MB? 1 GB? Is it unthinkable?
20 years ago, a 49 MB home page was unthinkable.
they won't be able to complain about low memory but their experience will be terrible every time they try to shove something horrible into the codebase
It’s as if everyone designed their website around the KPI of irritating your visitors and getting them to leave ASAP.
now: https://rd.nytimes.com
[0]: https://svelte.dev/
I’m afraid someone who wants to support professional journalism and agrees to pay ~$300/yr for an NYT subscription still gets most (all?) of this nonsense?
For every 1 engineer it seems like there are 5 PMs who need to improve KPIs somehow and thus decide auto playing video will improve metrics. It does. It also makes people hate using your website.
I would constantly try to push back against the bullshit they'd put on the page but no one really cares what a random engineer thinks.
I don't think there's any real way to solve this unless we either get less intrusive ad tech or news gets a better business model. Many sites don't even try with new business models, like local classifieds or local job boards. And good luck getting PMs to listen to an engineer talking about these things.
For now, the bloat remains.
You should probably tell someone so the knowledge doesn't die with you.
Bear in mind that any cut that reduces their CPM or rate of conversation to paid will have to also include an equivalent reduction in their staff.
I mean, they can absolutely try. That doesn't mean they should succeed.