Lizard Optimization
Fleeting- External reference:
- *L*earn how people are misusing the product.
- *Z*oom in on one behavior change at a time.
- *R*emove obstacles that we’ve put in front of users there.
- *D*etect unintended impacts because there will be lots of unintended impacts.
lizard’s men
[13:23.080 –> 13:29.080] Who had a situation where you thought, well, the user is really too stupid user product, be honest. [13:29.080 –> 13:34.080] Yeah, half of the room, half of the room, really. [13:34.080 –> 13:37.080] So it’s not unusual. There’s some research. [13:37.080 –> 13:42.080] And Scott Alexander published this research on his blog, Starslight Codex. [13:42.080 –> 13:45.080] I really love his ideas and I love his writing. [13:45.080 –> 13:50.080] And he wrote about this the demographical survey they’ve done. [13:50.080 –> 14:00.080] It was a combination of demographics and kind of some sociological studies where they’ve tried to qualify people, what their ethnicities, nationality and things like. [14:00.080 –> 14:02.080] And then kind of some conclusions try to compare that. [14:02.080 –> 14:10.080] And what they realized is that there’s a certain number of people that just like put idiotic answers there that you cannot explain. [14:10.080 –> 14:18.080] So a couple of people selected American as gender or entered American gender. [14:18.080 –> 14:20.080] OK, like gender politics are difficult now. [14:20.080 –> 14:23.080] You know, maybe that’s something that. [14:23.080 –> 14:30.080] But then some people selected Martian as an ethnicity. [14:30.080 –> 14:38.080] And then kind of what the conclusion is kind of is that there will be a number of people that are maybe confused when they use your software. [14:38.080 –> 14:41.080] They don’t know what you’re trying to do. [14:41.080 –> 14:46.080] They are maybe, you know, working with fat fingers on a small screen like I’m doing. [14:46.080 –> 14:51.080] So even though they wanted to kind of select the right thing, they select the wrong thing. [14:51.080 –> 14:55.080] Some people will have no mental capacity to understand what you’re doing. [14:55.080 –> 14:57.080] Some people will be distracted. [14:57.080 –> 15:02.080] Although they have mental capacity, they will temporarily be distracted from it and things like that. [15:02.080 –> 15:06.080] And then he pulled out this number called 4 percent and he called it the lizard man’s constant. [15:06.080 –> 15:12.080] And he said you have 4 percent lizard men in your in your population of users. [15:12.080 –> 15:16.080] Those are people who do unexplainable things because they follow their own logic. [15:16.080 –> 15:25.080] And he based this on kind of some research again where 4 percent of people believe that lizard men are ruling America or something like that. [15:25.080 –> 15:29.080] And kind of although this number is a joke number, of course, it’s a joke number. [15:29.080 –> 15:32.080] I think it’s kind of quite an interesting thing to think about. [15:32.080 –> 15:34.080] It’s not zero percent, is it?
[…]
[39:23.080 –> 39:25.080] But he misunderstood the message. [39:25.080 –> 39:26.080] The message was wrong. [39:26.080 –> 39:32.080] There was a mismatch between this person’s cognitive ability and my message. [39:32.080 –> 39:38.080] And I kind of tried to figure out, well, you know, why show people an error message at all? [39:38.080 –> 39:41.080] Why not kind of figure out what’s going on in a better way? [39:41.080 –> 39:43.080] And I loaded his text. [39:43.080 –> 39:47.080] And his text was 100 equal signs. [39:47.080 –> 39:48.080] Lizard!
serendipitously make wrong interaction go right
[11:08.080 –> 11:13.080] One of the products I built is reasonably popular. [11:13.080 –> 11:17.080] We had about 10 million active users last year. [11:17.080 –> 11:25.080] With 10 million active users, a certain percentage of people should not be allowed to use a keyboard, really. [11:25.080 –> 11:31.080] And this is my life now. Basically, I’m responding to emails like this all day long. [11:31.080 –> 11:36.080] This is an email I got a couple of weeks ago. I have no idea what it says. [11:36.080 –> 11:42.080] But what I can tell you is that this URL is completely invalid because it starts with a slash. [11:42.080 –> 11:48.080] And I know for a fact, because I built the website, that this web page does not exist. [11:48.080 –> 11:53.080] So this person is complaining in Arabic about the web page that doesn’t exist. [11:53.080 –> 12:00.080] And when I translated this, and I can even tell you what it is now without translating it because I get like a dozen emails like this a week. [12:00.080 –> 12:04.080] What this person’s done is they went on charge GPT. They asked charge GPT how to do something. [12:04.080 –> 12:12.080] Charge GPT hallucinated a URL to my website, but they recommended my software to this person. [12:12.080 –> 12:18.080] This person went to my website, got a broken page. The web page says this page does not exist, man. [12:18.080 –> 12:22.080] And he’s panicking now because charge GPT told him to do something that he can’t do. [12:22.080 –> 12:29.080] He can’t fall on the screen. So then he’s contacting me to say, well, your website is broken. It doesn’t work. [12:29.080 –> 12:36.080] As a result of this and a dozen other emails, I’ve actually built some decent intelligence into the 404 page of my website, [12:36.080 –> 12:44.080] which tries to guess what the people are trying to do when they come to the website and see a broken page and leads them there. [12:44.080 –> 12:47.080] So that’s my washing machine moment. [12:47.080 –> 12:53.080] But I think this is something that if it’s happening to me, it’s happening to all my competitors as well. [12:53.080 –> 13:02.080] And there’s a huge number of potential customers that we are all getting through hallucinated shit. [13:02.080 –> 13:10.080] And then one question is, are we just telling them, oh, you know, go away or are we helping them do the job that they’re supposed to do? [13:10.080 –> 13:14.080] So and again, you know, this is not really the only case.
[…]
[34:06.080 –> 34:10.080] And then kind of normally this is where most software stops. [34:10.080 –> 34:12.080] We tell the user you’ve done something wrong. [34:12.080 –> 34:14.080] Do it again. [34:14.080 –> 34:16.080] Try to explain what’s wrong. [34:16.080 –> 34:19.080] But I also have like this is the magic trick. [34:19.080 –> 34:21.080] Just log this. [34:21.080 –> 34:23.080] Just send it back to you. [34:23.080 –> 34:26.080] Log situations where users end up in a dead end. [34:26.080 –> 34:29.080] Send it back to yourself so you can analyze it. [34:29.080 –> 34:32.080] So I started logging this stuff.
decision making directly to the level of the people
[07:19.360 –> 07:29.520] You’ll have to figure that out. So what they try to do is basically push all the decision making directly to the level of the people [07:29.520 –> 07:37.360] who are in direct touch with the actual customer, not the customer representative, not the proxy of a customer representative, [07:37.360 –> 07:46.400] not the owner of the proxy of an area of a customer representative, but the actual people talking to the people buying stuff or using stuff. [07:46.440 –> 07:56.400] And one of the most inspiring stories in the book is when they started noticing a surge in broken washing machines in a particular rural region of China. [07:56.400 –> 08:04.920] It’s bizarre. As a statistic, it’s bizarre. You have kind of this specific washing machine being returned to the service centers [08:04.920 –> 08:10.440] much, much more frequently than all the other machines and than anywhere else in this specific region. [08:10.440 –> 08:16.080] So the first thing they thought about is there’s kind of some distribution fault. [08:16.080 –> 08:21.080] While you’re transporting the machines to this rural region, there’s a problem. They tried to kind of solve that. [08:21.080 –> 08:30.080] They couldn’t figure out anything wrong with it. Then they thought, well, there’s some kind of production problem for a factory that’s supplying this specific rural region. [08:30.080 –> 08:39.080] Nope, nothing there. And then they got people in the service centers to actually try to talk to people who return the machines and see what’s going on. [08:39.080 –> 08:49.080] And I took a bit of kind of persuading, but then one person admitted that instead of washing their clothes, they wash the potatoes before kind of going to the market. [08:49.080 –> 08:57.080] And so why is it? Well, you know, you wash the potatoes, they kind of absorb a bit of water. [08:57.080 –> 09:03.080] So they’re heavier. You can sell them for more money. Plus they look nicer. So, you know, people notice them in the market. [09:04.080 –> 09:13.080] And apparently one farmer in this rural region figured out that the lowest kind of power program on this machine doesn’t damage the potatoes that much. [09:13.080 –> 09:19.080] I don’t know why. Like, let’s not go into that. There’s a whole story behind that that we don’t know. [09:19.080 –> 09:25.080] But one person figured it out. That person talked to their friend, that person talked to their friend, that person talked to their friend. [09:25.080 –> 09:30.080] And all of a sudden you have these kind of thousands of farmers washing potatoes in this specific washing machine, [09:30.080 –> 09:36.080] this specific region where the machine is getting broken after a couple of washing cycles and they’re returning it. [09:36.080 –> 09:41.080] So what the higher people did at this point is when they figured that out, a couple of months later, [09:41.080 –> 09:47.080] there was a special machine for washing potatoes on the market and made a killing. [09:47.080 –> 09:52.080] And these people are one of the biggest brands in the world for kind of washing appliances, [09:52.080 –> 09:58.080] because they’re so quick figuring out what the customers need and iterating on that. And they’re allowed, they’re empowered to make these decisions.
bug vs feature -> expected vs unexpected
Let’s move out of the semantic debate around bug vs feature and focus on what matters, delivering value.
[16:08.080 –> 16:17.080] For example, about 20 years ago, I worked on an integration between a UK bookmaker, a gambling company and a US poker network. [16:17.080 –> 16:19.080] So the US poker network uses dollars. [16:19.080 –> 16:22.080] The UK gambling company uses pounds. [16:22.080 –> 16:23.080] No wonder about that. [16:23.080 –> 16:28.080] And in the poker network, dollar cents were chips. [16:28.080 –> 16:34.080] So you’d convert pounds into chips, which were dollar cents, send kind of chips back, which were dollar cents. [16:34.080 –> 16:41.080] So we had this weird situation where at that point and it’s coming to this kind of actually around again. [16:41.080 –> 16:43.080] A pound was zero fifty four dollars. [16:43.080 –> 16:46.080] A dollar was zero fifty four pounds. [16:46.080 –> 16:54.080] So what we had kind of after we deployed the software, we released it, it was tested, it was approved, it was accepted, it was live for about a month. [16:54.080 –> 17:10.080] And then somebody took one chip, one chip that was one dollar cent, converted it into pounds and ended up with one penny and then took that one penny, sent it back to dollar chips and ended up with two chips or two cents. [17:10.080 –> 17:13.080] And this is how inflation works. [17:13.080 –> 17:15.080] So that’s why computers cause inflation. [17:16.080 –> 17:21.080] So when this thing happened, of course, he didn’t stop there. [17:21.080 –> 17:23.080] He wrote a robot that automated this. [17:23.080 –> 17:28.080] And over two days he accumulated about twenty thousand pounds. [17:28.080 –> 17:34.080] And he withdrew the money because the person’s done nothing wrong. [17:34.080 –> 17:38.080] The bookmaker had to give him the money, but then everybody started screaming at everybody. [17:38.080 –> 17:41.080] The customer screamed at the customer representative. [17:41.080 –> 17:43.080] The customer representative screamed at the business analyst. [17:43.080 –> 17:45.080] The business analyst screamed at the product owner. [17:45.080 –> 17:47.080] The product owner screamed at the testers. [17:47.080 –> 17:49.080] The testers were screaming at developers. [17:49.080 –> 17:51.080] Everybody was screaming at everybody else. [17:51.080 –> 18:04.080] And this is the point where like a really funny thing started happening because the customer was complaining that we need to give them kind of some, that the company needs to kind of pay them back for damages. [18:04.080 –> 18:06.080] It was really weird. [18:06.080 –> 18:12.080] And the project manager basically claimed that everything is OK because like we’ve done what you wanted us to do. [18:12.080 –> 18:14.080] Like this was your spec. [18:14.080 –> 18:15.080] This was in the contract.
[…]
[18:15.080 –> 18:23.080] And then we started arguing or they started arguing whether this is a bug or a feature request to kind of fix. [18:23.080 –> 18:31.080] So and I originally was kind of under the fire there because I labeled it as a bug. [18:31.080 –> 18:35.080] And then a project manager, I worked with this project manager, wonderful, wonderful person. [18:35.080 –> 18:38.080] I don’t know if you’ve worked with somebody like that. [18:39.080 –> 18:44.080] It’s like after you quit your job and you see that person on the other side of the street, you just turn away and run. [18:44.080 –> 18:53.080] And so the project manager sat me and my colleague down and said, you are you are you are labeling too many things as bugs. [18:53.080 –> 18:54.080] So what does that mean? [18:54.080 –> 18:55.080] I mean, we have too many bugs. [18:55.080 –> 18:57.080] That’s like I’m not. [18:57.080 –> 18:58.080] It’s not my fault. [18:58.080 –> 19:01.080] I’m labeling them is the fault of kind of people putting bugs in. [19:01.080 –> 19:03.080] No, no, no. [19:03.080 –> 19:05.080] And she explained this really nicely. [19:05.080 –> 19:06.080] I like this definition. [19:06.080 –> 19:13.080] The only difference between a bug and a feature is who pays for it. [19:13.080 –> 19:16.080] So from now on, everything is a feature request. [19:16.080 –> 19:19.080] And we’re getting into this philosophical discussion now. [19:19.080 –> 19:20.080] It’s it’s idiotic. [19:20.080 –> 19:21.080] But, you know, that’s what it is. [19:21.080 –> 19:28.080] So there’s this kind of concept that we have that of bugs being wrong and features being right. [19:28.080 –> 19:29.080] But then what is wrong? [19:29.080 –> 19:30.080] What is right? [19:30.080 –> 19:35.080] I mean, you know, then we get into what Christian morality and perspectives of Saint Augustine. [19:35.080 –> 19:36.080] And who knows? [19:36.080 –> 19:38.080] You know, so let’s not do this.
[…]
[22:02.080 –> 22:10.080] So I think we need to move away as conceptually thinking about bugs and features is right or wrong. We can think about them as expected or unexpected. And that moves away from the whole kind of morality problem. [22:16.080 –> 22:21.080] As we can then think about, well, if we’re thinking about expected and unexpected, who’s expecting this? [23:37.080 –> 23:42.080] So if it does what the kind of user is expecting, the maker is expecting, that’s acceptable. That’s a feature. [23:44.080 –> 23:48.080] If it doesn’t do what you expect and doesn’t do what the user expects, that’s a bug. [23:48.080 –> 23:52.080] We can try to smuggle it as a feature request in a contract. [23:52.080 –> 23:54.080] It’s irrelevant, but it’s a bug.
[…]
[27:56.080 –> 27:58.080] moving away from the idea of right and wrong, [27:58.080 –> 28:01.080] but moving into kind of this idea of we have exploits, we have mismatches, [28:01.080 –> 28:03.080] we have mismatches that’s kind of a lot of gray area, [28:03.080 –> 28:08.080] is important because mismatches and exploits are hidden product opportunities. [28:08.080 –> 28:14.080] There’s a lot of stuff that can be discovered if we start looking for mismatches and exploits. [28:14.080 –> 28:18.080] And a lot of this stuff becomes really, really important for us [28:18.080 –> 28:21.080] to actually kind of keep our users engaged. [28:21.080 –> 28:24.080] Because when somebody’s exploiting your software, [28:24.080 –> 28:26.080] when somebody has a mismatch with your software, they’re there. [28:26.080 –> 28:28.080] They’re using your software. [28:28.080 –> 28:30.080] That’s the kind of important thing.
why the name lizard -> LZRD
[51:57.080 –> 51:59.080] One is to learn how people are misusing the product. [51:59.080 –> 52:02.080] The other is to zoom in on one behavior change at a time. [52:02.080 –> 52:06.080] The third is to remove obstacles that we’ve put in front of users there. [52:06.080 –> 52:12.080] And fourth is really to detect unintended impacts because there will be lots of unintended impacts. [52:12.080 –> 52:15.080] If you can’t remember this long, I have a shorter version of this. [52:15.080 –> 52:17.080] It’s called Lizard Alert. [52:17.080 –> 52:19.080] Zoom in, rescue them and double check. [52:19.080 –> 52:21.080] The initial letters are LZRD.