0

GitHub - stan-smith/FossFLOW: Make beautiful isometric infrastructure diagrams

preview

FossFLOW is a powerful, open-source Progressive Web App (PWA) for creating beautiful isometric diagrams. Built with React and the Isoflow (Now forked and published to NPM as fossflow) library, it runs entirely in your browser with offline support.

FossFLOW is a powerful, open-source Progressive Web App (PWA) for creating beautiful isometric diagrams. Built with React and the Isoflow (Now forked and published to NPM as fossflow) library, it runs entirely in your browser with offline support.

0

AI Is Quietly Rewriting Work—Here’s What You Need to Know

AI Is Quietly Rewriting Work—Here’s What You Need to Know gradientflow.substack.com
preview

Good (but scary) overview of state of AI in the og article, some points:

  • The transformation of work by AI is already underway, not a distant future event. The NBER model quantifies this as a potential 366% productivity boost coupled with a 23% employment reduction, with half the job displacement happening within five years. This underscores the urgency of being intentional about how AI reshapes work environments.

  • The future of AI in the workplace favors augmentation over replacement. Most workers prefer AI to automate repetitive tasks and to act as partners or coaches rather than substitutes. Enterprise AI strategies should therefore focus on complementing human skills, freeing teams to concentrate on creative and interpersonal value-driving activities.

  • AI adoption and impact are highly role-specific. While some jobs like logistics management face high automation risks, others like hands-on mechanics remain largely unaffected. Similarly, AI is democratizing expertise by leveling the playing field for freelancers, which challenges traditional premium skill valuations. Effective AI strategies require granular, nuanced understanding of these variations to maximize benefits and mitigate harms.

Good (but scary) overview of state of AI in the og article, some points:

  • The transformation of work by AI is already underway, not a distant future event. The NBER model quantifies this as a potential 366% productivity boost coupled with a 23% employment reduction, with half the job displacement happening within five years. This underscores the urgency of being intentional about how AI reshapes work environments.

  • The future of AI in the workplace favors augmentation over replacement. Most workers prefer AI to automate repetitive tasks and to act as partners or coaches rather than substitutes. Enterprise AI strategies should therefore focus on complementing human skills, freeing teams to concentrate on creative and interpersonal value-driving activities.

  • AI adoption and impact are highly role-specific. While some jobs like logistics management face high automation risks, others like hands-on mechanics remain largely unaffected. Similarly, AI is democratizing expertise by leveling the playing field for freelancers, which challenges traditional premium skill valuations. Effective AI strategies require granular, nuanced understanding of these variations to maximize benefits and mitigate harms.

0

California is set to become the first US state to manage power outages with AI

MIT Technology Review California is set to become the first US state to manage power outages with AI www.technologyreview.com
preview

“Automation,” the Boss says, eyes gleaming like a root shell. “AI will keep the lights on.”
So I wire up Genie - the latest in grid-bothering, outage-managing, decision-simulating snake oil.
The lusers? Giddy at the prospect of zero downtime. PFY suggests we let it loose on the test subnet. I suggest we don’t.
Genie parses, predicts, and promptly routes all non-critical loads to the CEO’s wine fridge. Network blips, Boss panics, solar array starts playing Tetris with the UPS.
PFY logs the event as “unexpected emergent behavior.” I call it a feature.
System’s stable, except for the part that isn’t.
“AI’s learning,” I tell the Boss. “So should you.”

“Automation,” the Boss says, eyes gleaming like a root shell. “AI will keep the lights on.”
So I wire up Genie - the latest in grid-bothering, outage-managing, decision-simulating snake oil.
The lusers? Giddy at the prospect of zero downtime. PFY suggests we let it loose on the test subnet. I suggest we don’t.
Genie parses, predicts, and promptly routes all non-critical loads to the CEO’s wine fridge. Network blips, Boss panics, solar array starts playing Tetris with the UPS.
PFY logs the event as “unexpected emergent behavior.” I call it a feature.
System’s stable, except for the part that isn’t.
“AI’s learning,” I tell the Boss. “So should you.”

0

Europe is looking for alternatives to US cloud providers

Ars Technica Europe is looking for alternatives to US cloud providers arstechnica.com
preview

cross-posted from: lemmy.world/post/27408560

cross-posted from: lemmy.world/post/27384100

The global backlash against the second Donald Trump administration keeps on growing. Canadians have boycotted US-made products, anti–Elon Musk posters have appeared across London amid widespread Tesla protests, and European officials have drastically increased military spending as US support for Ukraine falters. Dominant US tech services may be the next focus.

There are early signs that some European companies and governments are souring on their use of American cloud services provided by the three so-called hyperscalers. Between them, Google Cloud, Microsoft Azure, and Amazon Web Services (AWS) host vast swathes of the Internet and keep thousands of businesses running. However, some organizations appear to be reconsidering their use of these companies’ cloud services—including servers, storage, and databases—citing uncertainties around privacy and data access fears under the Trump administration.

“There’s a huge appetite in Europe to de-risk or decouple the over-dependence on US tech companies, because there is a concern that they could be weaponized against European interests,” says Marietje Schaake, a nonresident fellow at Stanford’s Cyber Policy Center and a former decadelong member of the European Parliament.

cross-posted from: lemmy.world/post/27408560

cross-posted from: lemmy.world/post/27384100

The global backlash against the second Donald Trump administration keeps on growing. Canadians have boycotted US-made products, anti–Elon Musk posters have appeared across London amid widespread Tesla protests, and European officials have drastically increased military spending as US support for Ukraine falters. Dominant US tech services may be the next focus.

There are early signs that some European companies and governments are souring on their use of American cloud services provided by the three so-called hyperscalers. Between them, Google Cloud, Microsoft Azure, and Amazon Web Services (AWS) host vast swathes of the Internet and keep thousands of businesses running. However, some organizations appear to be reconsidering their use of these companies’ cloud services—including servers, storage, and databases—citing uncertainties around privacy and data access fears under the Trump administration.

“There’s a huge appetite in Europe to de-risk or decouple the over-dependence on US tech companies, because there is a concern that they could be weaponized against European interests,” says Marietje Schaake, a nonresident fellow at Stanford’s Cyber Policy Center and a former decadelong member of the European Parliament.

0

How to Scale Agile Software Development with Technology and Lean

InfoQ How to Scale Agile Software Development with Technology and Lean www.infoq.com
preview

Highlights

The first scaling crisis happened in 1996, when Linus wrote that he was “buried alive in emails”. It was addressed by adopting a more modular architecture, with the introduction of loadable kernel modules, and the creation of the maintainers role, who support the contributors in ensuring that they implement the high standards of quality needed to merge their contributions.

The second scaling crisis lasted from 1998 to 2002, and was finally addressed by the adoption of BitKeeper, later replaced by Git. This distributed the job of merging contributions across the network of maintainers and contributors.

In both cases, technology was used to reduce the amount of dependencies between teams, help contributors keep a high level of autonomy, and make it easy to merge all those contributions back into the main repository, Bernhard said.

Technology can help reduce the need to communicate between teams whenever they have a dependency on another team to get their work done. Typical organizational dependencies, such as when a team relies on another team’s data, can be replaced by self-service APIs using the right technologies and architecture, Bernhard mentioned. This can be extended to more complicated dependencies, such as infrastructure provisioning, as AWS pioneered when they invented EC2, offering self-service APIs to spin up virtual servers, he added.

Another type of dependency is dealing with the challenge of merging contributions made to a similar document, whether it’s an illustration, a text, or source code, Bernhard mentioned. This has been transformed in the last 15 years by real-time collaboration software such as Google Docs and distributed versioning systems such as Git, he said.

Anyone trying to scale an agile organization should study lean thinking to benefit from decades of experience on how to lead large organizations while staying true to the spirit of the agile manifesto, he concluded.

Highlights

The first scaling crisis happened in 1996, when Linus wrote that he was “buried alive in emails”. It was addressed by adopting a more modular architecture, with the introduction of loadable kernel modules, and the creation of the maintainers role, who support the contributors in ensuring that they implement the high standards of quality needed to merge their contributions.

The second scaling crisis lasted from 1998 to 2002, and was finally addressed by the adoption of BitKeeper, later replaced by Git. This distributed the job of merging contributions across the network of maintainers and contributors.

In both cases, technology was used to reduce the amount of dependencies between teams, help contributors keep a high level of autonomy, and make it easy to merge all those contributions back into the main repository, Bernhard said.

Technology can help reduce the need to communicate between teams whenever they have a dependency on another team to get their work done. Typical organizational dependencies, such as when a team relies on another team’s data, can be replaced by self-service APIs using the right technologies and architecture, Bernhard mentioned. This can be extended to more complicated dependencies, such as infrastructure provisioning, as AWS pioneered when they invented EC2, offering self-service APIs to spin up virtual servers, he added.

Another type of dependency is dealing with the challenge of merging contributions made to a similar document, whether it’s an illustration, a text, or source code, Bernhard mentioned. This has been transformed in the last 15 years by real-time collaboration software such as Google Docs and distributed versioning systems such as Git, he said.

Anyone trying to scale an agile organization should study lean thinking to benefit from decades of experience on how to lead large organizations while staying true to the spirit of the agile manifesto, he concluded.

0

India Is Building an Open-Source Cloud Computing Effort

IEEE Spectrum India Is Building an Open-Source Cloud Computing Effort spectrum.ieee.org
preview

Will be interesting to see how it works out

The Indian nonprofit People+ai wants to fix this by creating an open and interoperable marketplace of cloud providers of all sizes. The Open Cloud Compute (OCC) project plans to use open protocols and standards to allow cloud providers of all sizes to offer their services on the network. It also plans to make it easy for customers to shift between offerings depending on their needs. People+ai held a hackathon on 20 September at People’s Education Society University (PES University) in Bengaluru to test out an early prototype of the platform.

Will be interesting to see how it works out

The Indian nonprofit People+ai wants to fix this by creating an open and interoperable marketplace of cloud providers of all sizes. The Open Cloud Compute (OCC) project plans to use open protocols and standards to allow cloud providers of all sizes to offer their services on the network. It also plans to make it easy for customers to shift between offerings depending on their needs. People+ai held a hackathon on 20 September at People’s Education Society University (PES University) in Bengaluru to test out an early prototype of the platform.

0

Improving platform resilience at Cloudflare through automation

Cloudflare Blog Improving platform resilience at Cloudflare through automation blog.cloudflare.com
preview

Highlights

Failure is an expected state in production systems, and no predictable failure of either software or hardware components should result in a negative experience for users. The exact failure mode may vary, but certain remediation steps must be taken after detection. A common example is when an error occurs on a server, rendering it unfit for production workloads, and requiring action to recover.

It can be tempting to rely on the expertise of world-class engineers to remediate these faults, but this would be manual, repetitive, unlikely to produce enduring value, and not scaling.

The commonality of lower-priority failures makes it obvious when the response required, as defined in runbooks, is “toilsome”. To reduce this toil, we had previously implemented a plethora of solutions to automate runbook actions such as manually-invoked shell scripts, cron jobs, and ad-hoc software services. These had grown organically over time and provided solutions on a case-by-case basis, which led to duplication of work, tight coupling, and lack of context awareness across the solutions.

A good solution would not allow only the SRE team to auto-remediate, it would empower the entire company. The key to adding self-healing capability was a generic interface for all teams to self-service and quickly remediate failures at various levels: machine, service, network, or dependencies.

Temporal is a durable execution platform which is useful to gracefully manage infrastructure failures such as network outages and transient failures in external service endpoints. This capability meant we only needed to build a way to schedule “workflow” tasks and have Temporal provide reliability guarantees.

After a workflow is validated in the staging environment, we can then do a full release to production. It seems obvious, but catching simple configuration errors before releasing has saved us many hours in development/change-related-task time.

Building a system that is maintained by several SRE teams has allowed us to iterate faster, and rapidly tackle long-standing problems. We have set ambitious goals regarding toil elimination and are on course to achieve them, which will allow us to scale faster by eliminating the human bottleneck.

Highlights

Failure is an expected state in production systems, and no predictable failure of either software or hardware components should result in a negative experience for users. The exact failure mode may vary, but certain remediation steps must be taken after detection. A common example is when an error occurs on a server, rendering it unfit for production workloads, and requiring action to recover.

It can be tempting to rely on the expertise of world-class engineers to remediate these faults, but this would be manual, repetitive, unlikely to produce enduring value, and not scaling.

The commonality of lower-priority failures makes it obvious when the response required, as defined in runbooks, is “toilsome”. To reduce this toil, we had previously implemented a plethora of solutions to automate runbook actions such as manually-invoked shell scripts, cron jobs, and ad-hoc software services. These had grown organically over time and provided solutions on a case-by-case basis, which led to duplication of work, tight coupling, and lack of context awareness across the solutions.

A good solution would not allow only the SRE team to auto-remediate, it would empower the entire company. The key to adding self-healing capability was a generic interface for all teams to self-service and quickly remediate failures at various levels: machine, service, network, or dependencies.

Temporal is a durable execution platform which is useful to gracefully manage infrastructure failures such as network outages and transient failures in external service endpoints. This capability meant we only needed to build a way to schedule “workflow” tasks and have Temporal provide reliability guarantees.

After a workflow is validated in the staging environment, we can then do a full release to production. It seems obvious, but catching simple configuration errors before releasing has saved us many hours in development/change-related-task time.

Building a system that is maintained by several SRE teams has allowed us to iterate faster, and rapidly tackle long-standing problems. We have set ambitious goals regarding toil elimination and are on course to achieve them, which will allow us to scale faster by eliminating the human bottleneck.

0

Finnish Startup Wants to Build 100x Faster CPUs

IEEE Spectrum Finnish Startup Wants to Build 100x Faster CPUs spectrum.ieee.org
preview

Valtonen’s goal is to put CPUs back in their rightful, ‘central’ role. In order to do that, he and his team are proposing a new paradigm. Instead of trying to speed up computation by putting 16 identical CPU cores into, say, a laptop, a manufacturer could put 4 standard CPU cores and 64 of Flow Computing’s so-called parallel processing unit (PPU) cores into the same footprint, and achieve up to 100 times better performance. Valtonen and his collaborators laid out their case at the IEEE Hot Chips conference in August.

Valtonen’s goal is to put CPUs back in their rightful, ‘central’ role. In order to do that, he and his team are proposing a new paradigm. Instead of trying to speed up computation by putting 16 identical CPU cores into, say, a laptop, a manufacturer could put 4 standard CPU cores and 64 of Flow Computing’s so-called parallel processing unit (PPU) cores into the same footprint, and achieve up to 100 times better performance. Valtonen and his collaborators laid out their case at the IEEE Hot Chips conference in August.

0

[PATCH v2 0/8] Enable build system on macOS hosts - Daniel Gomez via B4 Relay

preview

This is interesting and potentially useful for anyone, who works in the corp which does not allow Linux laptops, but you can get your hands on Macs.

This is interesting and potentially useful for anyone, who works in the corp which does not allow Linux laptops, but you can get your hands on Macs.

0

How We Built the Internet

How We Built the Internet every.to
preview

How We Built the Internet

Metadata

Highlights

The internet is a universe of its own.

The infrastructure that makes this scale possible is similarly astounding—a massive, global web of physical hardware, consisting of more than 5 billion kilometers of fiber-optic cable, more than 574 active and planned submarine cables that span a over 1 million kilometers in length, and a constellation of more than 5,400 satellites offering connectivity from low earth orbit (LEO).

“The Internet is no longer tracking the population of humans and the level of human use. The growth of the Internet is no longer bounded by human population growth, nor the number of hours in the day when humans are awake,” writes Geoff Huston, chief scientist at the nonprofit Asia Pacific Network Information Center.

As Shannon studied the structures of messages and language systems, he realized that there was a mathematical structure that underlied information. This meant that information could, in fact, be quantified.

Shannon noted that all information traveling from a sender to a recipient must pass through a channel, whether that channel be a wire or the atmosphere.

Shannon’s transformative insight was that every channel has a threshold—a maximum amount of information that can be delivered reliably to a sender.

Kleinrock approached AT&T and asked if the company would be interested in implementing such a system. AT&T rejected his proposal—most demand was still in analog communications. Instead, they told him to use the regular phone lines to send his digital communications—but that made no economic sense.

What was exceedingly clever about this suite of protocols was its generality. TCP and IP did not care which carrier technology transmitted its packets, whether it be copper wire, fiber-optic cable, or radio. And they imposed no constraints on what the bits could be formatted into—video text, simple messages, or even web pages formatted in a browser.

David Clark, one of the architects of the original internet, wrote in 1978 that “we should … prepare for the day when there are more than 256 networks in the Internet.”

Fiber was initially laid down by telecom companies offering high-quality cable television service to homes. The same lines would be used to provide internet access to these households. However, these service speeds were so fast that a whole new category of behavior became possible online. Information moved fast enough to make applications like video calling or video streaming a reality.

And while it may have been the government and small research groups that kickstarted the birth of the internet, its evolution henceforth was dictated by market forces, including service providers that offered cheaper-than-ever communication channels and users that primarily wanted to use those channels for entertainment.

In 2022, video streaming comprised nearly 58 percent of all Internet traffic. Netflix and YouTube alone accounted for 15 and 11 percent, respectively.

At the time, Facebook users in Asia or Africa had a completely different experience to their counterparts in the U.S. Their connection to a Facebook server had to travel halfway around the world, while users in the U.S. or Canada could enjoy nearly instantaneous service. To combat this, larger companies like Google, Facebook, Netflix, and others began storing their content physically closer to users through CDNs, or “content delivery networks.”

Instead of simply owning the CDNs that host your data, why not own the literal fiber cable that connects servers from the United States to the rest of the world?

Most of the world’s submarine cable capacity is now either partially or entirely owned by a FAANG company—meaning Facebook (Meta), Amazon, Apple, Netflix, or Google (Alphabet).

Google, which owns a number of sub-sea cables across the Atlantic and Pacific, can deliver hundreds of terabits per second through its infrastructure.

In other words, these applications have become so popular that they have had to leave traditional internet infrastructure and operate their services within their own private networks. These networks not only handle the physical layer, but also create new transfer protocols —totally disconnected from IP or TCP. Data is transferred on their own private protocols, essentially creating digital fiefdoms.

SpaceX’s Starlink is already unlocking a completely new way of providing service to millions. Its data packets, which travel to users via radio waves from low earth orbit, may soon be one of the fastest and most economical ways of delivering internet access to a majority of users on Earth. After all, the distance from LEO to the surface of the Earth is just a fraction of the length of subsea cables across the Atlantic and Pacific oceans.

What is next?

How We Built the Internet

Metadata

Highlights

The internet is a universe of its own.

The infrastructure that makes this scale possible is similarly astounding—a massive, global web of physical hardware, consisting of more than 5 billion kilometers of fiber-optic cable, more than 574 active and planned submarine cables that span a over 1 million kilometers in length, and a constellation of more than 5,400 satellites offering connectivity from low earth orbit (LEO).

“The Internet is no longer tracking the population of humans and the level of human use. The growth of the Internet is no longer bounded by human population growth, nor the number of hours in the day when humans are awake,” writes Geoff Huston, chief scientist at the nonprofit Asia Pacific Network Information Center.

As Shannon studied the structures of messages and language systems, he realized that there was a mathematical structure that underlied information. This meant that information could, in fact, be quantified.

Shannon noted that all information traveling from a sender to a recipient must pass through a channel, whether that channel be a wire or the atmosphere.

Shannon’s transformative insight was that every channel has a threshold—a maximum amount of information that can be delivered reliably to a sender.

Kleinrock approached AT&T and asked if the company would be interested in implementing such a system. AT&T rejected his proposal—most demand was still in analog communications. Instead, they told him to use the regular phone lines to send his digital communications—but that made no economic sense.

What was exceedingly clever about this suite of protocols was its generality. TCP and IP did not care which carrier technology transmitted its packets, whether it be copper wire, fiber-optic cable, or radio. And they imposed no constraints on what the bits could be formatted into—video text, simple messages, or even web pages formatted in a browser.

David Clark, one of the architects of the original internet, wrote in 1978 that “we should … prepare for the day when there are more than 256 networks in the Internet.”

Fiber was initially laid down by telecom companies offering high-quality cable television service to homes. The same lines would be used to provide internet access to these households. However, these service speeds were so fast that a whole new category of behavior became possible online. Information moved fast enough to make applications like video calling or video streaming a reality.

And while it may have been the government and small research groups that kickstarted the birth of the internet, its evolution henceforth was dictated by market forces, including service providers that offered cheaper-than-ever communication channels and users that primarily wanted to use those channels for entertainment.

In 2022, video streaming comprised nearly 58 percent of all Internet traffic. Netflix and YouTube alone accounted for 15 and 11 percent, respectively.

At the time, Facebook users in Asia or Africa had a completely different experience to their counterparts in the U.S. Their connection to a Facebook server had to travel halfway around the world, while users in the U.S. or Canada could enjoy nearly instantaneous service. To combat this, larger companies like Google, Facebook, Netflix, and others began storing their content physically closer to users through CDNs, or “content delivery networks.”

Instead of simply owning the CDNs that host your data, why not own the literal fiber cable that connects servers from the United States to the rest of the world?

Most of the world’s submarine cable capacity is now either partially or entirely owned by a FAANG company—meaning Facebook (Meta), Amazon, Apple, Netflix, or Google (Alphabet).

Google, which owns a number of sub-sea cables across the Atlantic and Pacific, can deliver hundreds of terabits per second through its infrastructure.

In other words, these applications have become so popular that they have had to leave traditional internet infrastructure and operate their services within their own private networks. These networks not only handle the physical layer, but also create new transfer protocols —totally disconnected from IP or TCP. Data is transferred on their own private protocols, essentially creating digital fiefdoms.

SpaceX’s Starlink is already unlocking a completely new way of providing service to millions. Its data packets, which travel to users via radio waves from low earth orbit, may soon be one of the fastest and most economical ways of delivering internet access to a majority of users on Earth. After all, the distance from LEO to the surface of the Earth is just a fraction of the length of subsea cables across the Atlantic and Pacific oceans.

What is next?

0

Incantations

Incantations josvisser.substack.com
preview

Incantations

Metadata

Highlights

The problem with incantations is that you don’t understand in what exact circumstances they work. Change the circumstances, and your incantations might work, might not work anymore, might do something else, or maybe worse, might do lots of damage. It is not safe to rely on incantations, you need to move to understanding.

Incantations

Metadata

Highlights

The problem with incantations is that you don’t understand in what exact circumstances they work. Change the circumstances, and your incantations might work, might not work anymore, might do something else, or maybe worse, might do lots of damage. It is not safe to rely on incantations, you need to move to understanding.

0

How much are your 9's worth?

How much are your 9's worth? hross.substack.com
preview

How much are your 9’s worth?

Metadata

Highlights

All nines are not created equal. Most of the time I hear an extraordinarily high availability claim (anything above 99.9%) I immediately start thinking about how that number is calculated and wondering how realistic it is.

Human beings are funny, though. It turns out we respond pretty well to simplicity and order.

Having a single number to measure service health is a great way for humans to look at a table of historical availability and understand if service availability is getting better or worse. It’s also the best way to create accountability and measure behavior over time…

… as long as your measurement is reasonably accurate and not a vanity metric.

Cheat #1 - Measure the narrowest path possible.

This is the easiest way to cheat a 9’s metric. Many nines numbers I have seen are various version of this cheat code. How can we create a narrow measurement path?

Cheat #2 - Lump everything into a single bucket.

Not all requests are created equal.

Cheat #3 - Don’t measure latency.

This is an availability metric we’re talking about here, why would we care about how long things take, as long as they are successful?!

Cheat #4 - Measure total volume, not minutes.

Let’s get a little controversial.

In order to cheat the metric we want to choose the calculation that looks the best, since even though we might have been having a bad time for 3 hours (1 out of every 10 requests was failing), not every customer was impacted so it wouldn’t be “fair” to count that time against us.

Building more specific models of customer paths is manual. It requires more manual effort and customization to build a model of customer behavior (read: engineering time). Sometimes we just don’t have people with the time or specialization to do this, or it will cost to much to maintain it in the future.

We don’t have data on all of the customer scenarios. In this case we just can’t measure enough to be sure what our availability is.

Sometimes we really don’t care (and neither do our customers). Some of the pages we build for our websites are… not very useful. Sometimes spending the time to measure (or fix) these scenarios just isn’t worth the effort. It’s important to focus on important scenarios for your customers and not waste engineering effort on things that aren’t very important (this is a very good way to create an ineffective availability effort at a company).

Mental shortcuts matter. No matter how much education we try, it’s hard to change perceptions of executives, engineers, etc. Sometimes it is better to pick the abstraction that helps people understand than pick the most accurate one.

Data volume and data quality are important to measurement. If we don’t have a good idea of which errors are “okay” and which are not, or we just don’t have that much traffic, some of these measurements become almost useless (what is the SLO of a website with 3 requests? does it matter?).

What is your way of cheating nines? ;)

How much are your 9’s worth?

Metadata

Highlights

All nines are not created equal. Most of the time I hear an extraordinarily high availability claim (anything above 99.9%) I immediately start thinking about how that number is calculated and wondering how realistic it is.

Human beings are funny, though. It turns out we respond pretty well to simplicity and order.

Having a single number to measure service health is a great way for humans to look at a table of historical availability and understand if service availability is getting better or worse. It’s also the best way to create accountability and measure behavior over time…

… as long as your measurement is reasonably accurate and not a vanity metric.

Cheat #1 - Measure the narrowest path possible.

This is the easiest way to cheat a 9’s metric. Many nines numbers I have seen are various version of this cheat code. How can we create a narrow measurement path?

Cheat #2 - Lump everything into a single bucket.

Not all requests are created equal.

Cheat #3 - Don’t measure latency.

This is an availability metric we’re talking about here, why would we care about how long things take, as long as they are successful?!

Cheat #4 - Measure total volume, not minutes.

Let’s get a little controversial.

In order to cheat the metric we want to choose the calculation that looks the best, since even though we might have been having a bad time for 3 hours (1 out of every 10 requests was failing), not every customer was impacted so it wouldn’t be “fair” to count that time against us.

Building more specific models of customer paths is manual. It requires more manual effort and customization to build a model of customer behavior (read: engineering time). Sometimes we just don’t have people with the time or specialization to do this, or it will cost to much to maintain it in the future.

We don’t have data on all of the customer scenarios. In this case we just can’t measure enough to be sure what our availability is.

Sometimes we really don’t care (and neither do our customers). Some of the pages we build for our websites are… not very useful. Sometimes spending the time to measure (or fix) these scenarios just isn’t worth the effort. It’s important to focus on important scenarios for your customers and not waste engineering effort on things that aren’t very important (this is a very good way to create an ineffective availability effort at a company).

Mental shortcuts matter. No matter how much education we try, it’s hard to change perceptions of executives, engineers, etc. Sometimes it is better to pick the abstraction that helps people understand than pick the most accurate one.

Data volume and data quality are important to measurement. If we don’t have a good idea of which errors are “okay” and which are not, or we just don’t have that much traffic, some of these measurements become almost useless (what is the SLO of a website with 3 requests? does it matter?).

What is your way of cheating nines? ;)

0

The one about the web developer job market

The one about the web developer job market www.baldurbjarnason.com
preview

cross-posted from: group.lt/post/1926151

Don’t read if you don’t want to ruin your day.

The One About The Web Developer Job Market

Metadata

Highlights

Many organisations are also resorting to employee-hostile strategies to increase employee churn, such as forced Return-To-Office policies.

Finding a non-bullshit job is likely only going to get harder.

• Finding effective documentation, information, and training is likely to get harder, especially in specialised topics where LLMs are even less effective than normal.

as soon as you start to try to predict the second or third order consequences things very quickly get remarkably difficult.

In short, futurists are largely con artists.

you need to do something Note: ah… the famous “you need to do something” This is not a one-off event but has turned into a stock market driven movement towards reducing the overall headcount of the tech industry.

What this means is that when the bubble ends, as all bubbles must, the job market is likely to collapse even further.

The stock market loves job cuts

Activist investors see it as an opportunity to lower developer compensation

Management believes they can replace most of these employees with LLM-based automation

Discovering whether it’s true or not is actually quite complicated as it, counter-intuitively, doesn’t depend on the degree of LLM functionality but instead depends entirely on what organisations, managers, and executives are using software projects for.

Since, even in the best case scenario of the most optimistic prediction of LLM power, you’re still going to need to structure a plan for the code and review it, the time spent on code won’t drop to zero. But if you believe in the best-case prediction, a 20-40% improvement in long term productivity sounds reasonable, if a bit conservative.

The alternate world-view, one that I think is much more common among modern management, is that the purpose of software development is churn.

None of these require that the software be free or even low on defects. The project doesn’t need to be accessible or even functional for a majority of the users. It just needs to look good when managers, buyers, and sales people poke at it.

The alternate world-view, which I think I can demonstrate is dominant in web development at least, is that software quality does not matter. Production not productivity is what counts. Up until now, the only way to get production and churn has been to focus on short-term developer experience, often at the expense of the long term health of the project, but the innovation of LLMs is that now you can get more churn, more production, with fewer developers.

This means that it doesn’t matter who is correct or not in their estimate of how well these tools work.

You aren’t going to notice the issue as an end-user. From your perspective the system is working perfectly.

Experienced developers will edit out the issues without thinking about it, focusing on the time-saving benefit of generating the rest. Inexperienced developers won’t notice the issues and think they’ve just saved a lot of time, not realising they’ve left a ticking time bomb in the code base.

The training data favours specific languages such as Python and JavaScript.

• The same tool that enhances their productivity by 20-30% might also be outright harmful to a junior developer’s productivity, once you take the inevitable and eventual corrections and fixes into account.

From the job market perspective, all that improved and safe LLM-based coding tools would mean is more job losses.

Because manager world-views are more important than LLM innovation.

If the technology is what’s promised, the churn world-view managers will just get more production, more churn, with even fewer developers. The job market for developers will decline.

they will still use the tech to increase production, with fewer developers, because software quality and software project success isn’t what they’re looking for in software development. The job market for developers will decline.

The more progress you see in the automation, the fewer of us they’ll need. It isn’t a question of the nature of the improvement, but of the attitudes of management.

Even if that weren’t true, technical innovations in programming generally don’t improve project or business outcomes.

The odds of a project’s success are dictated by user research, design, process, and strategy, not the individual technological innovations in programming. Rapid-Application-Development tools, for example, didn’t shift outcomes in meaningful ways.

What matters is whether the final product works and improves the business it was made for. Business value isn’t solely a function of code defects. Technical improvements that address code defects are necessary, but not sufficient.

tech industry management is firmly convinced that less is more when it comes to employing either.

Whether you’re a bear or a bull on LLMs, we as developers are going to get screwed either way, especially if we’re web developers.

Most web projects shipped by businesses today are broken, but businesses rarely seem to care.

Most websites perform so badly that they don’t even finish loading on low-end devices, even when business outcomes directly correlate with website performance, such as in ecommerce or ad-supported web media.

The current state of web development is as if most Windows apps released every year simply failed to launch on 20-40% of all supported Windows installs.

If being plausible is all that matters, then that’s the literal, genuine, core strength of an LLM.

This is a problem for the job market because if all that matters to these organisations is being seen plausibly chasing cutting-edge technology – that the actual business outcomes don’t matter – then the magic of LLMs mean that you don’t actually need that many developers to do that for much, much less money.

Web media is a major employer, both directly and indirectly, of web developers. If a big part of the web media industry is collapsing, then that’s an entire sector that isn’t hiring any of the developers laid off by Google, Microsoft, or the rest. And the people they aren’t hiring will still be on the job market competing with everybody else who wouldn’t have even applied to work in web media.

The scale of LLM-enabled spam production outstrips the ability of Bing or Google to counter it.

But it gets even worse as every major search engine provider on the market is all-in on replacing regular keyword search with chatbots and LLM-generated summaries that don’t drive any traffic at all to their sources.

It’s reasonable to expect that the job market is unlikely to ever fully bounce back, due to the collapse of web media alone.

Experience in Node or React is not a reliable signifier of an ability to work on successful Node or React projects because most Node or React projects aren’t even close to being successful from a business perspective. Lack of experience in Node or React – such as a background in other frameworks or vanilla JS – conversely isn’t a reliable signifier that the developer won’t be a successful hire.

Lower pay, combined with the information asymmetry about employer dysfunction, would then lead to more capable workers leaving the sector, either to run their own businesses – a generally dysfunctional web development sector is likely to have open market opportunities – or leave the industry altogether. This would exacerbate the job market’s dysfunctions even further, deepening the cycle.

The first thing to note is that, historically, whenever management adopts an adversarial attitude towards labour, the only recourse labour has is to unionise.

As employees, we have nothing to lose from unionising. That’s the first consequence of management deciding that labour is disposable.

Diversifying your skills has always been a good idea for a software developer. Learning a new language gives you insight into the craft of programming that is applicable beyond that language specifically.

But the market for developer training in general has collapsed.

Some of it is down to the job market. Why invest in training if tech cos aren’t hiring you anyway? Why invest in training your staff if you’re planning on replacing them with LLM tools anyway?

After all, as Amy Hoy wrote in 2016:

Running a biz is a lot less risky than having a job, because 1000 customers is a lot less fragile than 1 employer.

The tech industry has “innovated” itself into a crisis, but because the executives aren’t the ones out looking for jobs, they see the innovations as a success.

The rest of us might disagree, but our opinions don’t count for much.

But what we can’t do is pretend things are fine.

Because they are not.

Thoughts?

cross-posted from: group.lt/post/1926151

Don’t read if you don’t want to ruin your day.

The One About The Web Developer Job Market

Metadata

Highlights

Many organisations are also resorting to employee-hostile strategies to increase employee churn, such as forced Return-To-Office policies.

Finding a non-bullshit job is likely only going to get harder.

• Finding effective documentation, information, and training is likely to get harder, especially in specialised topics where LLMs are even less effective than normal.

as soon as you start to try to predict the second or third order consequences things very quickly get remarkably difficult.

In short, futurists are largely con artists.

you need to do something Note: ah… the famous “you need to do something” This is not a one-off event but has turned into a stock market driven movement towards reducing the overall headcount of the tech industry.

What this means is that when the bubble ends, as all bubbles must, the job market is likely to collapse even further.

The stock market loves job cuts

Activist investors see it as an opportunity to lower developer compensation

Management believes they can replace most of these employees with LLM-based automation

Discovering whether it’s true or not is actually quite complicated as it, counter-intuitively, doesn’t depend on the degree of LLM functionality but instead depends entirely on what organisations, managers, and executives are using software projects for.

Since, even in the best case scenario of the most optimistic prediction of LLM power, you’re still going to need to structure a plan for the code and review it, the time spent on code won’t drop to zero. But if you believe in the best-case prediction, a 20-40% improvement in long term productivity sounds reasonable, if a bit conservative.

The alternate world-view, one that I think is much more common among modern management, is that the purpose of software development is churn.

None of these require that the software be free or even low on defects. The project doesn’t need to be accessible or even functional for a majority of the users. It just needs to look good when managers, buyers, and sales people poke at it.

The alternate world-view, which I think I can demonstrate is dominant in web development at least, is that software quality does not matter. Production not productivity is what counts. Up until now, the only way to get production and churn has been to focus on short-term developer experience, often at the expense of the long term health of the project, but the innovation of LLMs is that now you can get more churn, more production, with fewer developers.

This means that it doesn’t matter who is correct or not in their estimate of how well these tools work.

You aren’t going to notice the issue as an end-user. From your perspective the system is working perfectly.

Experienced developers will edit out the issues without thinking about it, focusing on the time-saving benefit of generating the rest. Inexperienced developers won’t notice the issues and think they’ve just saved a lot of time, not realising they’ve left a ticking time bomb in the code base.

The training data favours specific languages such as Python and JavaScript.

• The same tool that enhances their productivity by 20-30% might also be outright harmful to a junior developer’s productivity, once you take the inevitable and eventual corrections and fixes into account.

From the job market perspective, all that improved and safe LLM-based coding tools would mean is more job losses.

Because manager world-views are more important than LLM innovation.

If the technology is what’s promised, the churn world-view managers will just get more production, more churn, with even fewer developers. The job market for developers will decline.

they will still use the tech to increase production, with fewer developers, because software quality and software project success isn’t what they’re looking for in software development. The job market for developers will decline.

The more progress you see in the automation, the fewer of us they’ll need. It isn’t a question of the nature of the improvement, but of the attitudes of management.

Even if that weren’t true, technical innovations in programming generally don’t improve project or business outcomes.

The odds of a project’s success are dictated by user research, design, process, and strategy, not the individual technological innovations in programming. Rapid-Application-Development tools, for example, didn’t shift outcomes in meaningful ways.

What matters is whether the final product works and improves the business it was made for. Business value isn’t solely a function of code defects. Technical improvements that address code defects are necessary, but not sufficient.

tech industry management is firmly convinced that less is more when it comes to employing either.

Whether you’re a bear or a bull on LLMs, we as developers are going to get screwed either way, especially if we’re web developers.

Most web projects shipped by businesses today are broken, but businesses rarely seem to care.

Most websites perform so badly that they don’t even finish loading on low-end devices, even when business outcomes directly correlate with website performance, such as in ecommerce or ad-supported web media.

The current state of web development is as if most Windows apps released every year simply failed to launch on 20-40% of all supported Windows installs.

If being plausible is all that matters, then that’s the literal, genuine, core strength of an LLM.

This is a problem for the job market because if all that matters to these organisations is being seen plausibly chasing cutting-edge technology – that the actual business outcomes don’t matter – then the magic of LLMs mean that you don’t actually need that many developers to do that for much, much less money.

Web media is a major employer, both directly and indirectly, of web developers. If a big part of the web media industry is collapsing, then that’s an entire sector that isn’t hiring any of the developers laid off by Google, Microsoft, or the rest. And the people they aren’t hiring will still be on the job market competing with everybody else who wouldn’t have even applied to work in web media.

The scale of LLM-enabled spam production outstrips the ability of Bing or Google to counter it.

But it gets even worse as every major search engine provider on the market is all-in on replacing regular keyword search with chatbots and LLM-generated summaries that don’t drive any traffic at all to their sources.

It’s reasonable to expect that the job market is unlikely to ever fully bounce back, due to the collapse of web media alone.

Experience in Node or React is not a reliable signifier of an ability to work on successful Node or React projects because most Node or React projects aren’t even close to being successful from a business perspective. Lack of experience in Node or React – such as a background in other frameworks or vanilla JS – conversely isn’t a reliable signifier that the developer won’t be a successful hire.

Lower pay, combined with the information asymmetry about employer dysfunction, would then lead to more capable workers leaving the sector, either to run their own businesses – a generally dysfunctional web development sector is likely to have open market opportunities – or leave the industry altogether. This would exacerbate the job market’s dysfunctions even further, deepening the cycle.

The first thing to note is that, historically, whenever management adopts an adversarial attitude towards labour, the only recourse labour has is to unionise.

As employees, we have nothing to lose from unionising. That’s the first consequence of management deciding that labour is disposable.

Diversifying your skills has always been a good idea for a software developer. Learning a new language gives you insight into the craft of programming that is applicable beyond that language specifically.

But the market for developer training in general has collapsed.

Some of it is down to the job market. Why invest in training if tech cos aren’t hiring you anyway? Why invest in training your staff if you’re planning on replacing them with LLM tools anyway?

After all, as Amy Hoy wrote in 2016:

Running a biz is a lot less risky than having a job, because 1000 customers is a lot less fragile than 1 employer.

The tech industry has “innovated” itself into a crisis, but because the executives aren’t the ones out looking for jobs, they see the innovations as a success.

The rest of us might disagree, but our opinions don’t count for much.

But what we can’t do is pretend things are fine.

Because they are not.

Thoughts?

0

COBOL Programmers are Back In Demand. Seriously. – Communications of the ACM

preview

Highlights

COBOL remains crucial to businesses and institutions around the world.

It is estimated $3 trillion in daily commerce flows through COBOL systems, while 95% of ATM swipes and 80% of in-person banking transactions rely on COBOL code.

when unemployment claims suddenly spiked due to the pandemic, these archaic systems could not keep up, which means that benefits are not being distributed.

The spike in unemployment claims exposed another new problem: there is no one around to repair these legacy systems.

Although a few universities still offer COBOL courses, the number of people studying it today is extremely small.

COBOL Cowboys’ business model is more akin to the gig economy rather than to that of the companies at which these industry veterans spent their careers. It is staffed with mostly older freelancers, everyone is an independent consultant, and there is no promise of any work. The company’s slogan is “not our first rodeo.”

“A lot of us want to spend time with our grandkids, but we also want to keep busy.”

Hinshaw was in contact with the state of New Jersey at the beginning of the current crisis, and quickly saw that the unemployment claims issue wasn’t a back-end problem. Every claim that was sent to the host (the back-end mainframe) was processed.

“They all have the same problem on the front end,” says Hinshaw, adding that these organizations’ Web sites were not designed to handle that kind of volume, while the back-end mainframes typically can.

IBM, which sold many of the mainframes on which COBOL systems run, has been scrambling to launch initiatives in order to meet the urgent need for COBOL programmers to address the overloaded unemployment systems.

While these measures should eventually help to alleviate the shortage in COBOL programming expertise, it is clear that the past approach of “if it isn’t broken, don’t fix it” has contributed to the current problem.

Are you learning COBOL already? ;)

Highlights

COBOL remains crucial to businesses and institutions around the world.

It is estimated $3 trillion in daily commerce flows through COBOL systems, while 95% of ATM swipes and 80% of in-person banking transactions rely on COBOL code.

when unemployment claims suddenly spiked due to the pandemic, these archaic systems could not keep up, which means that benefits are not being distributed.

The spike in unemployment claims exposed another new problem: there is no one around to repair these legacy systems.

Although a few universities still offer COBOL courses, the number of people studying it today is extremely small.

COBOL Cowboys’ business model is more akin to the gig economy rather than to that of the companies at which these industry veterans spent their careers. It is staffed with mostly older freelancers, everyone is an independent consultant, and there is no promise of any work. The company’s slogan is “not our first rodeo.”

“A lot of us want to spend time with our grandkids, but we also want to keep busy.”

Hinshaw was in contact with the state of New Jersey at the beginning of the current crisis, and quickly saw that the unemployment claims issue wasn’t a back-end problem. Every claim that was sent to the host (the back-end mainframe) was processed.

“They all have the same problem on the front end,” says Hinshaw, adding that these organizations’ Web sites were not designed to handle that kind of volume, while the back-end mainframes typically can.

IBM, which sold many of the mainframes on which COBOL systems run, has been scrambling to launch initiatives in order to meet the urgent need for COBOL programmers to address the overloaded unemployment systems.

While these measures should eventually help to alleviate the shortage in COBOL programming expertise, it is clear that the past approach of “if it isn’t broken, don’t fix it” has contributed to the current problem.

Are you learning COBOL already? ;)

0

Google Cloud accidentally deletes a financial institution account due to ‘unprecedented misconfiguration’

the Guardian Google Cloud accidentally deletes UniSuper’s online account due to ‘unprecedented misconfiguration’ www.theguardian.com
preview

cross-posted from: feddit.it/post/7752642

A week of downtime and all the servers were recovered only because the customer had a proper disaster recovery protocol and held backups somewhere else, otherwise Google deleted the backups too

Google cloud ceo says “it won’t happen anymore”, it’s insane that there’s the possibility of “instant delete everything”

cross-posted from: feddit.it/post/7752642

A week of downtime and all the servers were recovered only because the customer had a proper disaster recovery protocol and held backups somewhere else, otherwise Google deleted the backups too

Google cloud ceo says “it won’t happen anymore”, it’s insane that there’s the possibility of “instant delete everything”

0

Architecture Does Not Emerge - a Conversation with Tracy Bannon

InfoQ Architecture Does Not Emerge - a Conversation with Tracy Bannon www.infoq.com
preview

Good before going to bed on Friday evening :)

Some things I have noted:

Until this day, I say I’ve done three things: insert levels of indirection, trade off space and time, and three, try to get my clients to tell me what they really want.

If you’re doing something with ERP systems, well, first of all, I apologize and feel bad for you in life, but that’s a whole other conversation.

Let’s think somebody ultimately is paying for this thing to be built. Somebody somewhere has a vision on what they want it to be. There are humans who will eventually be using it. It needs to meet those business or mission needs. It has to start with that.

what bothers me is when people make implicit assumptions and don’t make them explicit.

“The decisions that you make upfront are the ones that,” and this is paraphrasing, “the ones that are too expensive and you cannot change later.”

But I’m talking about when you make a decision, write it down, make an architectural decision record. It can be itty bitty, itty bitty. But just Tracy made this decision today. Context, we don’t have a license for that and it’s going to take eight months to get the new license or acquisition or whatever else.

The other thing that struck me about your example about the people doing the two front ends, and I’m going to use this to loop back into the conversation about developers and architects, is that they don’t understand, or appear not to understand, that in the global perspective, by picking two different UI designs, you’ve made the programmer hiring decision harder.

I’ve had a lot of contentious conversations with folks who say, “I’m a solution architect.” Well, what technologies do you dabble with? Well, I haven’t touched code in about 20 years.

It’s, if you’re going to be an architect and have that mindset, you need to be able to go from the boiler room to the boardroom. You need to be able to communicate, but it also means that in order to be trusted, you have to bring your chops to the table.

I think lack of taxonomy is probably one of the killers in any organization. You and I don’t agree on what that word means. And with that comes so much nuance and with that comes muscle memory and process issues. That’s something that just drives me crazy on a daily basis.

So I am a real junkie when it comes to people talking about how processes don’t work. Well, let’s have an hour conversation. Let’s map out how it’s actually working. I’m not talking about Lean Six Sigma values. I’m talking about, let’s find the waste.

One of the things that I would have people to take away is the need to constantly be considering how you can decouple or loosely couple things, because that aids in the longevity. If you think about even electronics and things that you bought in your house, the big integrated front of your dishwasher, I now have to replace the entire dishwasher.

Because back in my day, full stack meant you are actually worth your salt.

My way is not always the right way. Don’t let anybody hear that, but it’s true.

Good before going to bed on Friday evening :)

Some things I have noted:

Until this day, I say I’ve done three things: insert levels of indirection, trade off space and time, and three, try to get my clients to tell me what they really want.

If you’re doing something with ERP systems, well, first of all, I apologize and feel bad for you in life, but that’s a whole other conversation.

Let’s think somebody ultimately is paying for this thing to be built. Somebody somewhere has a vision on what they want it to be. There are humans who will eventually be using it. It needs to meet those business or mission needs. It has to start with that.

what bothers me is when people make implicit assumptions and don’t make them explicit.

“The decisions that you make upfront are the ones that,” and this is paraphrasing, “the ones that are too expensive and you cannot change later.”

But I’m talking about when you make a decision, write it down, make an architectural decision record. It can be itty bitty, itty bitty. But just Tracy made this decision today. Context, we don’t have a license for that and it’s going to take eight months to get the new license or acquisition or whatever else.

The other thing that struck me about your example about the people doing the two front ends, and I’m going to use this to loop back into the conversation about developers and architects, is that they don’t understand, or appear not to understand, that in the global perspective, by picking two different UI designs, you’ve made the programmer hiring decision harder.

I’ve had a lot of contentious conversations with folks who say, “I’m a solution architect.” Well, what technologies do you dabble with? Well, I haven’t touched code in about 20 years.

It’s, if you’re going to be an architect and have that mindset, you need to be able to go from the boiler room to the boardroom. You need to be able to communicate, but it also means that in order to be trusted, you have to bring your chops to the table.

I think lack of taxonomy is probably one of the killers in any organization. You and I don’t agree on what that word means. And with that comes so much nuance and with that comes muscle memory and process issues. That’s something that just drives me crazy on a daily basis.

So I am a real junkie when it comes to people talking about how processes don’t work. Well, let’s have an hour conversation. Let’s map out how it’s actually working. I’m not talking about Lean Six Sigma values. I’m talking about, let’s find the waste.

One of the things that I would have people to take away is the need to constantly be considering how you can decouple or loosely couple things, because that aids in the longevity. If you think about even electronics and things that you bought in your house, the big integrated front of your dishwasher, I now have to replace the entire dishwasher.

Because back in my day, full stack meant you are actually worth your salt.

My way is not always the right way. Don’t let anybody hear that, but it’s true.

0

SSH over HTTPS

preview

sometimes i see this in “very secured” servers as well. so check web configs, especially if you takeover the server management from somebody else ;)

sometimes i see this in “very secured” servers as well. so check web configs, especially if you takeover the server management from somebody else ;)