Narrator: You're listening to the humans of DevOps podcast, a

podcast focused on advancing the humans of DevOps through skills,

knowledge, ideas, and learning, or the skil framework.

Max Howell: We see this all over the place with open source. I

think a lot of the time the people who make successful open

source, making a hobby project, some kind, something they need,

and then it becomes an essential piece of internet

infrastructure.

Eveline Oehrlich: In December of 2004, a group of IIT students

from DAP University in Copenhagen has launched a open

source beer, the one that a version was described by those

students as a great tasting, energetic beer, and its world

first open source beer. It's based on a classic ale brewing

tradition. But with added Guarana for a natural energy

boost. The aim was to see what happens when an open source

structure is applied to a universally known product like

beer. Hello there. This is evolutionarily ish. We are not

here to talk about beer. But we're here to talk about open

source. And I brought this introduction because I think it

fits in multiple ways you will find out yourself. But I'm a

beer Brewer by Hobby so that caught my attention. Our podcast

title today is open source, Abreu and t. Now a few more

details, which is very different 97% of commercial code contains

open source and I actually learned this from our guest.

When looking at some of the tech trends such as containerization,

like Docker and Kubernetes, originally developed at Google

and released as open source in 2014. Big data like Apache Spark

with Hadoop or Kafka, a lot of open source is out there. Again,

as I said, Welcome to humans of DevOps podcast, I'm Avalon early

Chief Research Officer at DevOps Institute. Today, we have a very

special guest, and I'm excited to have him with us for multiple

reasons. Max Howell, CEO of T dot x, y, z. Hello, Max. Welcome

to our podcast.

Max Howell: Thank you. I'm pleased to be here.

Eveline Oehrlich: Yes, excited for you to be with us. Tell us

the first thing really I'm curious about is your journey

into an across open source.

Max Howell: Yeah, well, it's a fairly good story. So I did a

chemistry degree because I thought I wanted to be a

scientist. And in the process of doing that degree, and then

doing a year in the industry, I discovered that actually, it was

really not for me, I was working with this device that measured

surface tension for like various chemical solutions. And I

realized that if I kept working there, like in 10 years, I would

still be using that single machine and making very minor

contributions to the world. So I fell into this sort of

depressing funk. And I discovered open source.

Programming was always something that I had considered a hobby.

My dad taught me when I was six, which, you know, very young, but

I only used it to like make video games essentially, like

most kids do when they learned to program. And I never really

considered it for a career until my career was on the rocks. And

I didn't know what I was doing myself. So propelled into open

source by installing Linux. And I loved the community, I loved

how everyone was super passionate about what they were

doing, and just doing it because they wanted to do something that

changed the world or improved their workflow or just just

making cool things. And I ended up working on a few apps with

people from all over the world and it was just like nothing

else. For me. Honestly, it's, it's difficult to find it as

good as it was when I first got into it. And we were all on IRC

together. And doing that led me to getting a job in the

industry, which you know, it's it's kind of luck in many ways.

But there was a company in London that used one of the apps

I was working on and invited me to go and interview and it

wasn't really as much of an interview as just having a chat.

So I actually got myself into the industry. And a few years

later after that, you know, I kept doing open source and so I

created homebrew, which is one of the biggest open source

projects of all time. At this point, and I'm sure you'll have

some more questions about that.

Eveline Oehrlich: Yep. Yep, keep going, because I will come,

we'll come back to that.

Max Howell: So I created homebrew because at the time, we

were making all these different apps. I was at last firm in

London, and we made six different apps. One was for

Linux and Mac and Windows, we have an iPhone app, and an

Android app, and even this Blackberry app. And we we build

them all on Mac, because Mac was like the unified platform, time,

and what it really has become the platform for development

now. But this was 2008. And at that time, like, developers

hadn't really decided what platform they're using, it was

all over the place, it was still still a lot of people using

Windows for development, which is not as common nowadays. And

in the our office was an awful lot of people on Linux, and I

was one of the few who converted to Mac. Because Apple had a bad

rep. With developers. It's hard to believe now. But well, it's

easier to believe over the last few years. But there was

certainly a period where everyone in development used to

back. But that was new. And we were using it because it was the

platform that you could do Android Dev, you could do Linux

Dev, you could have a Windows VM running. So we could do all six

platforms quite easily. And like the package management solutions

were the acceptable, they weren't great. And they

certainly weren't designed for developers, their impact

managers turned out with Linux. And in many ways, they were the

thing which defined the Linux flavor. And it's still still the

case. But I kind of felt that they were designed for sis ops

and DevOps, but not development. So I wanted to build something

that was more for that. And so I got on with it. And after a few

months, I realized that it was kind of neat. So I open sourced

it. And then it took a few months before anyone noticed it.

But then when he got noticed, it took off, like amazingly

quickly, huge amounts of contribution, but a lot of

excitement, because I've managed to tap into something which

people wanted, needed, especially with this burgeoning

developer platform, which was what the Mac was becoming. And

yeah, so here we are, like, almost 14 years later. And yeah,

it's it's an enormous project. At this point. It's very hard to

meet people that haven't heard of it or used it.

Eveline Oehrlich: You know, I have watched your speak at Web

Summit in 2022. And you talked about something there called the

Nebraska problem. And And honestly, I've been in

infrastructure operations for a long time, but I'm not a

developer. So maybe that's why I've never really heard. I've

understood, I've heard of it, but I really don't understand or

didn't understand what it actually meant. So can you give

us a quick explanation of what is meant by the Nebraska

problem? And how it relates to the open source? And if

possible, any examples you have seen recently? Maybe?

Max Howell: Yeah, sure. So what we call the Nebraska problem is

the open source funding problem. And we call it that because

there's this famous XKCD comic, you'll see it trotted out

whenever there's any discussion about open source funding

issues, and represent it shows like this tower of blocks, and

it represents all modern digital infrastructure. And then near

the bottom of the towers is precarious, little pillar, and

it's holding the whole thing up. And there's an arrow pointing to

it, which says, this open source package maintained by some

someone from Nebraska, since 2003, thanklessly. Are a

suddenly experienced this with homebrew, like, it was kind of

the first time I'd really experienced it, because even

though I'd done some relatively successful open source before

that, well, you know, I did work full time on a few projects

before that, but I was I didn't have a job and didn't know what

I was going to do with myself. I was still living with my

parents. So it was to do it. With homebrew, I had to have two

full time jobs. One was paid and one which didn't. And I took

some time off here and there so I could work on homebrew full

time. But I never had enough money saved up to work on it for

a long period of time, but I couldn't abandon it because it

was well it was my easily at the time like my greatest creation.

I was extremely proud of it but also people depended on it. And

I Couldn't let them down, I just didn't see how I could. So I

work at, you know, my office job, and often was doing some

homebrew stuff to be honest. And then in the evenings and the

weekends, yeah, I didn't really have a social life and just

worked on homebrew. And that was like, wanted or needed, but it

wasn't sustainable. And in the end, I did burn out on it. And I

haven't really worked on homebrew since 2016. I passed it

to the community, which, you know, I was, I'm lucky that I

could, that suddenly other people turned up. And this is

how open source is and how it should work. Of course,

community is essential to open source. And so it was good that

I could hand it over, but I can personally keep working on it.

And we see this all over the place with open source, I think

a lot of the time the people who make successful open source are

just making a hobby project for some kind, something they need.

And then it becomes an essential piece of internet

infrastructure. And they're stuck in a situation where they

either like, abandon it. And then that's not what they want

to do. They can't afford to work on it as much as they need to.

While they try to find funding, and let's face it most most

funding doesn't doesn't work very well for open source. A lot

of the time then Nebraska projects is what we call them,

like famous one couple of years ago was locked for J which is a

logging piece of logging software for Java applications.

And there was a massive exploit found in it where you could root

servers just by typing stuff in if the stuff you talked to him

would go through the logging software in some capacity. And

no one had really heard of this package, because it was deep in

the stack. And the sounds themselves often works, right.

It's like these essential pieces of software have been built. But

over time, they've just been buried on the things that are

built on top of them. And you got really mature pieces of

software, which still need to be maintained and everyone's

forgotten exist. A lot of for Jay was a great example of that.

And they fixed the bug. And they asked that maybe they could get

some funding in future so that they could afford to spend more

time actually maintaining the software and maybe making sure

it doesn't have these security holes. And I don't think they

ever received anything, because once the bug was fixed, everyone

forgot about them again. And then more recently, there was

core J S, nine D in downloads in is since it was released, the

core of the base of every node app that exists, every you know,

everything was built on Node or uses Node, and an essential

piece of infrastructure really, but the person who maintains it

is giving up because they're just fed up with the fact that

they don't get any funding. Everyone uses this thing.

Everyone is extremely entitled about how they treat open

source. I'm afraid to say it's often true. And he's just like,

well, I can't afford it I'm giving up. And what's going to

happen, I don't know, it's not a good thing to use at the

infrastructure of and, you know, I, I've done loads of open

source. And the truth is, I really should just been doing

open source all these years like I created one homebrew with the

small amount of free time I had. People I myself shouldn't have

to make a choice between working on software that improves the

world and is beneficial to like all layers of the software

stack. Or some of the some of the jobs I did were not

particularly beneficial to the world, really. So that's the

Nebraska problem.

Narrator: Skill up it learning is the perfect online

destination to learn about DevOps and digital

transformation, anytime, anywhere. Our digital learning

platform provides you with all the resources you need to

upskill and learn about these important topics, including

expert led course videos, access to certification, prep courses,

and a community of supportive peers. Our subscription based

model makes it easy for you to gain the skills needed for

success in the modern workplace. Visit DevOps institute.com now

and explore our different learning plans.

Eveline Oehrlich: Wow, wow, quite quite involved. quite

complicated. And when you started out just said, you know,

this noble cause of contributing to the world really with what

you have done with homebrew. We'll get to T XYZ in a minute.

But I wanted to ask a couple other questions. Because of the

topic of open source, there's still some people seem to not

understand that perspective. So that's why I was very, very

interested in your thoughts and your journey there. Let's switch

a little bit to the benefits in terms of of leveraging open

source. You know, take it from either the benefits into from a

developer perspective collaborating and into the open

source community, or for the benefits of the community over

in its large meaning the world right. Talk a little bit to us

about what you see the benefits are of open for open, not open

force, but open source.

Max Howell: Yeah, open source is really interesting. I don't

think that human history is really got any other equivalent

examples. Or markets usually up here, because there's an

economic incentive there. Like I don't know if anything else like

this. But I've been thinking about this quite a lot recently.

And I remember how in the 90s, the entire software stack was

basically owned by Microsoft. And, you know, there was

numerous issues with this. But certainly, as the Internet

became a thing, it became more and more clear that Microsoft

was holding the Internet back, because Internet Explorer was

the only browser that anyone used, and they didn't want to

advance it. They didn't care, too, there are no real

incentive. And it was this kind of attitude, which led to open

source taking more and more of a hold. In the end, like now, it's

difficult to imagine a world where open source isn't the base

of every stack, like we say, 97% of all commercial software has

open source involved in it in some capacity, and like, this is

a really free percent that doesn't, I'd like to see what

those source bases and understand how they get away

with not having any open source at all. It Over time we, we

built the open source app, and we swapped off the Microsoft

stack. And it became like the point where developers realized

that they not only gained like all this functionality for free,

but also they were getting these step ups in productivity. And

proficiency is open source is usually designed to be like

these tiny little self contained pieces that you slop together,

and you build on top of each other. And then you've got all

this slight advanced functionality. But in the

process of replacing Microsoft with open source, we never

transferred all that money that makes over making while they

were trillions of dollars in. And instead, we just sort of

ended up with Freebase without real any maintenance on it,

which, you know, we're trying to solve AT T realized the the

value of open sources. People throw out ideas and the good

ones stick, there's no marketing behind it. There's no like

someone's trying to sell you something that actually they

know isn't a good fit for you. You put some open source out

there. And if it works for the developers of this world, they

adopt it? Well, this is sort of people's world, they adopt it.

And then over time, if it continues to fit, it builds

until it becomes like a mature block in the stack.

Eveline Oehrlich: So with that there are challenges as well.

What are some challenges you would say? Again, from multiple

perspectives, take it from the developer, take it from the

community or take it from the software world of vendors out

there?

Max Howell: Well, I think vendors have the highest risk

right now, right? But if you're building an app, and it has 6000

dependencies, either, there's no way you can vet that. And be

sure that all those dependencies are secure. First off, also

without any malicious insertions of any kind. It's one thing

we're going to try and fix with what we're doing at T. But I do

wonder how, you know, if I had to report to someone and say

that our app is secure, we we can guarantee that I don't know

how I'd say that with a straight face 6000 depths in there. So

there's a lot of risk there for vendors. I think developers,

most of them just enjoy open source existing because it

really does make computing more enjoyable in general, especially

now. Like 14 years ago when I made homebrew, like the amount

of people doing open sources so much smaller. And now you can't

go a day without dozens of new things being released

continuously. And some of those are great. But in most

developers feel bad that they just consume this open source

and don't give back but I don't see how they can effectively

give back right because you can try and spot through a few open

source steps here and there, but we're all using 1000s and 1000s

of open source projects. And it's just infeasible without

some layer of automation to contribute to those properly.

But open source developers are the ones that I feel for but you

know, most of them are just do it, they start off by doing side

projects, it really is, it's like, it's one of the beautiful

things about software engineering is that it's so

cheap and easy. All you need is time, that if you're gonna lay

there, and you got some time, you can, you can create some

open source. But what happens with all the open source devs,

I've talked to who are, have got successful projects is that

after a while, it's a huge amount of work. And you just

can't abandon that. You have to somehow find the time to do it.

People on myself, we've, the way I approached, it was always to

try and make it so that as much as possible, it was robust. And

then but if you did break, it would tell the user what they

can do to try and fix it themselves. And they'll make

sure the documentation was as good as I can get it. And that

would reduce the burden on me as much as I could, as much as

possible. And then, you know, if you if you're lucky, a community

turns up, mostly open source, I've done communities that turn

up, homebrew was

Eveline Oehrlich: really quite an exception. Hmm, fascinating.

Absolutely fascinating. So again, at the Web Summit, you

said, I love this vision you painted, you said, you want to

support those who are passionate on open source development, but

make sure that they get rewarded. And you actually call

it I love this to digital revolution for developers. What

do you mean by that? And then we'll get to T XYZ, because I

think there is a relationship. So tell us about the digital

revolution for the developers?

Max Howell: Well, let's face it, in the sense of web two, as we

call it, which was, you know, mid 2000s, till, kind of

recently, I'd say, a huge amount of money was made by people

building on top of open source. And it was gradual, you know,

like the way open source became so essential to what we're

building and software was gradual. And I think a lot of

these essential pieces, if they were built by hackers who were

building them for other hackers, and then when web two turned up

and figured out how to monetize all that, there wasn't any

transition of wealth. And so that's why it's a digital

revolution. These huge, mega billion dollar companies that

have never really given any substantial amount to the open

source that they depend on. It's not right, like we were okay

with it as it happened, because it was gradual. But now we like

hanging on. You know, why am I working two full time jobs when

Microsoft has billions and billions of dollars coming in

every month? Doesn't make sense. So what we're trying to build a

T is a way to successfully remunerate how open source is

funded. There's been lots of attempts, sponsorship, bounties,

things like this, but we see that as being reward favorites.

And some people make a lot of money with sponsorship. But the

truth is, there's millions of open source packages. And most

of them are Nebraska projects that nobody knows about. React

doesn't need to be funded, because Facebook, pay for it

essentially, which is another thing we want to fix instantly.

I don't believe that these huge corporations should have so much

agenda over how these open source projects that the

community needs to figure out what makes sense for open

source. Facebook could just like close down react tomorrow. I'd

want with what we're building a tee is for these developers at

Facebook and Microsoft who do work on open source or who are

just extremely talented people to be able to quit and work on

open source full time because the pay is exactly the same or

better. Even, frankly, we're building an economy on code. And

if you make a bunch of successful open source packages,

providing you're using the T protocol, you could be much

wealthier than you are currently as a software developer.

Eveline Oehrlich: Interesting. So how can developers get

involved if I am sure we have a few of those who are going ooh,

I love this. I love hearing this. How can they get involved?

Max Howell: Well, four months ago I released the tea package

manager so tea is the successor to brew you know I've stopped

working on brew in mid to 2010s. And I didn't stop thinking about

what could be better about broom. But I didn't really have

a good incentive I feel to make another brutal I'd already done

it. And I remember the tireless hours that I spent working on it

robust, defying it, filling out the package graph, answering

people's support questions, and just like trying to make sure it

was a successful and useful piece of software. But I didn't

really want to go and do that again. I think I like doing

things once usually, but But 18 months ago, while I was looking

into web three, and crypto, which basically was the first

time I've looked into this stuff, because I've never been

particularly interested in it. I thought Bitcoin was pretty

impressive. As a wonderful mystery story to it also pretty

ingenious, how it works, and how it has successfully become so

large, even though it's not controlled. But I never really

thought that there was much more that was interesting there for

me. But as I was diving into web three stuff, I started to see

the different contracts were actually pretty interesting, I

had this moment of inspiration while messing around an open sea

with NF Ts. Because NF T's allowed you to well view the

open sea digital contract anyway forced any repurchases to put

10% back to the original creator. I was like, Oh, so you

can write a digital contract that funnels money automatically

to different entities. And I realized suddenly that the

package graph of all open source was that kind of, if you put a

digital contract on it anyway, you could funnel money to all

the dependencies, all the packages. So all these Nebraska

projects could get like little bits of token from any insertion

of token near the top. And that's how it works, right? Like

people sponsor these big projects of near the top. And

it's all the ones underneath that that are failing, have are

maintained by people without any things. And they're essential

pieces of the digital infrastructure. Without them,

everything would collapse. I like everything being that

fragile is scary, like people remember that for a long time?

How it could all collapse any moment, but somehow it doesn't.

And we're just waiting for a bigger claps really. But also,

like, think about what would happen if it was all funded

correctly. People myself, like make, like occasionally big and

important projects. But we have to do it in our spare time. What

if I was working on that full time? And then, you know, we

find security holes and open SSL every two months? Well, if it

was funded correctly, so that well, either of us so could

really afford to hire people who could, you know, like kind of

like a corporation is what we're thinking things light, each of

these big projects will have their own dowels on the T

protocol. And they can operate them. Similar to a kind of

corporation, they can hire people using the token, they can

split the rewards that they're getting from the tee protocol

across everyone is contributing to open SSL. So if you open a

pull request, you could end up getting paid for that pull

request. Because the DAO is a set of smart contracts that

figure out how to channel the token based on the governance

structure that those projects have chosen. CL I came up with

the idea, like build a package manager and build it on top of

blockchain. So that open source can be remunerated, then we

managed to raise 80 million, and we've been building it since. So

I released the package manager itself four months ago, got six

sales and styles now got some really passionate users. I think

it's special. And if anyone is interested in getting involved

with what we're doing, that's the that's the place to start.

Currently, what we got is github.com/t. XYZ and you can

check it out, install it, and I think it's a better package

manager the brew it's, it's more than a package manager, really,

the ideas I had over the years for how the package manager

could be improved. I realized that it's a unique piece of

software, right? It sits underneath all other tooling.

There's so many things that you can do, which nobody's really

tried before. I don't really get it I think perhaps package

managers just aren't very sexy. Is the truth and don't many

people that are interested in working on them but for some

reason I've always enjoyed it. I love getting into that sort of

area of it.

Eveline Oehrlich: It is a brilliant idea. You are

brilliant. I am honored to talk to you. If I come up with a name

for this that I will share it with You but it is. It's

certainly a super cool and absolutely honorable, really

honorable thing you're doing. I love the idea. I'm, I'm very

grateful. Now I have one closing question. Hmm, maybe it involves

beer drinking, but what do you do for fun? What do you do for

fun if you don't do these brilliant things and manage

these fantastic things? You are You were you were talking about?

Max Howell: Yeah, well, so like, programming was always a

hobbies. Switch when I switch to doing it for career did become

kind of what I did one as well. But certainly, you know, I used

to live in the UK. I live in the States now. And I used to love

going to pubs friends. So there was some beer drinking in my

palace for sure. But I got a son now he's 10 months. And so

there's certainly less beer in my life at the moment.

Otherwise, for fun, so Blimey, I do like video games here and

there on Sunday not playing them as much as I used to. And I like

hiking or going out or camping isn't some of the things I

really enjoy.

Eveline Oehrlich: Excellent. Well, if you make it to my part

of the world, I'm happy to take you out for beer. We got some

pretty good beer in Germany. So yes, yes. Well, this has been

fantastic. Thank you so much Max. We have been talking to max

Howell CEO of T dot XYZ and brilliant thought leader in the

topic of open source Mexican, thank you so much for joining me

today on humans of DevOps podcast. Thank you very much.

Humans of DevOps podcast is produced by DevOps Institute.

Our audio production team includes Julia pape, Daniel

Newman, Schultz and Brendan Lee, shout out to my colleagues who

do a fantastic job. I'm humans of DevOps podcast, Executive

Producer, evolutionarily, if you would like to join us on a

podcast, please contact us at humans of DevOps podcast at

DevOps institute.com. I'm evolutionarily talk to you soon.

Narrator: Thanks for listening to this episode of the humans of

DevOps podcast. Don't forget to join our global community to get

access to even more great resources like this. Until next

time, remember, you are part of something bigger than yourself.

You belong