Why Your AI Learning Projects Keep Fizzling Out

[00:00] The grand vision is you can learn anything with a book. [00:03] Anything that you ever thought was too hard to learn, we could at least get you started and you can feel like it is achievable. [00:09] Why does it need to be a separate app? Why isn't ChatGPT just the ideal way to learn new things? [00:13] A real learning platform has to be built as a learning platform. LOMs are built to be a general tool, but given that they were not built as learning tools at their core, they're missing a lot. [00:36] - Nir, welcome to the show. [00:38] Thanks for having me. [00:39] So for people who don't know, you are the co-founder and CEO of Oboe. [00:46] Uh, which I don't know what your one liner is, but, um, from my perspective, it's like a, it's an AI learning app that makes, um, like one-off courses for you, like on demand, basically. Um, it's, it's pretty cool. Uh, you can tell me where I'm wrong there. Uh, but it's, it's a really, it's a really good app. I think it fits with a lot of stuff you've been thinking about. Um, uh, prior to, prior to doing Oboe, you were, um, most well known for being an every writer sometimes. Of course. [01:16] and global head of audiobooks at Spotify. And before that, you were the co-founder of Anchor. Excited to have you on the show. [01:24] Thanks for having me. And yeah, I was thinking back on the writing that I had done for every... I actually do think it's very on brand for Oboe because a lot of what I'd written was about

[01:33] trying to make people realize that they can learn and understand things that [01:37] they probably were intimidated by and thought were too hard to learn. [01:40] I love that you're doing this because I think it's... [01:43] I think AI is just so good for learning. It has expanded my mind in so many different ways. And I want to ask the tough question up front, which is, why does it need to be a separate app? Why isn't ChatGPT just the ideal way to learn new things? [01:57] Look, LLMs are incredible. I use them. [02:01] Um, [02:02] Now, honestly, in every aspect of my life, as I know you do too, and I think it's becoming more and more the case for... [02:09] everybody who realizes how powerful it is. But [02:12] LOMs are built to be a general [02:15] tool, like the most universally general tool that you could possibly have. I've spent a long time thinking about learning. I've spent a long time thinking about it both from the perspective of [02:25] an entrepreneur who wants to build products in the space. [02:28] but also as a person who uses these products day in and day out to actually teach myself things. I teach myself things a lot and I have for years and... [02:36] What I fundamentally believe is that [02:39] Uh, [02:40] while chat is extremely [02:42] powerful for learning. It is not the primary way that people do learn. [02:47] Most learning is passive. It's not active. It doesn't require lean and participation. Most of the learning that you've done in your life has been through the consumption of content passively. [02:56] with active engagement. [02:58] every once in a while. [02:59] most of the learning that you've done in your life has been multimodal, right? It has not been

[03:04] sitting and consuming and engaging with content in one particular way. It's been done through [03:09] piecing together different formats online, for instance, right? Every single day, I'm willing to bet that you do this and everybody listening does this. You get curious about something, you read about something in the news, you identify something that you don't know too much about, and you go down rabbit holes on the internet to try and learn those things. [03:26] And you don't just default to ChatGPT or other LLMs, despite the fact that they're really, really powerful. You may use them as one tool in the arsenal of tools that you use to learn, [03:35] but they are just one modality. People learn through [03:38] multimodality right so you'll start with chat gpt and then you'll google some stuff and you'll end up on wikipedia and you'll go to youtube [03:45] And you're not alone in doing that because... [03:48] Billions of people do that every single day. Like there are billions of people who use the internet. [03:52] to learn every day and to piece together. [03:56] a learning experience. [03:58] using these different formats and these different platforms. [04:02] And so back to your question, I think LLMs are a piece of the puzzle. [04:06] And I think they enable... [04:08] an important piece of the puzzle, but given that they were not built as learning tools, [04:12] at their core, they're missing a lot. And I think that a real learning platform has to be built as a learning platform. As I believe that a real [04:21] fill in the blank platform. [04:23] that many of your guests are probably building on the show. [04:27] Those have to be built by people who are focused on a particular use case. [04:30] That's really interesting. I want to go back to something you said, which sort of shocked me, which is that most learning is passive, not active, or like,

[04:37] basically good learning is passive, not active, is did I get you right? Because I would have assumed the opposite. [04:43] And so like my mental model of like how learning works is you like, [04:48] At least for me, it's very much learning in context. I'm like, I need to do a specific thing. And so I want to like understand... [04:53] how to do that thing and how it might fit into all the other things that I know and want to do, which feels very active to me. I do spend a lot of time reading, which I guess is passive and I am learning stuff, but I just hadn't really, if I had to bet what you would say, I would have said, [05:06] learning happens as an active thing. [05:09] Well, I think what we're talking about are probably two different dimensions, right? What you're talking about is intentionality and... [05:15] objective. Where does the objective come into learning? I do believe that the best learning happens through high intent, high agency, [05:24] and significant objective-driven motivation. [05:28] And that's actually a thing that we talk about a lot as far as the product goes. [05:32] Most of the use cases that people today already are using Ovo for are objective oriented. People come in and they say, [05:39] I know what I want to accomplish. I want to be good at X or I want to be able to [05:43] uh, [05:44] take out a mortgage to buy a house. I have no idea what that is. Or I want to be able to do [05:49] gain this particular skill so that I can up-skill at work. [05:52] or I want to pass this particular test, all of those fall under this umbrella of high intent, [05:57] objective-driven motivation. [06:00] Um, [06:02] That... [06:04] I think is separate from the dimensionality of how you actually go about learning the thing. So I do think that high intent is very important. It is what motivates people the most. And that's why it's important to build a product that taps into whatever their objective is and tries to embrace a path that gets them to their objective as quickly as possible and lets them see that it is feasible for them to hit that goal.

[06:22] But then the question of how do you actually present the material to them [06:25] goes to the heart of a different dimension, which is [06:29] what I think is passive versus active or single modal versus multimodal. If you think back to school and you think about the best teachers that you had most of the time, [06:38] that you were learning, you were learning from them teaching to you with an opinion about how to teach you, you were sitting and consuming them talking or consuming the reading or whatever it was, you were not actively participating in the conversation. That's not to say that active participation is not very important. It's one of the modalities that I think reinforces... [06:56] a lot of learning that people are able to achieve. But it's not the primary mechanism, right? And what LLMs do is they basically put the onus on the user, [07:04] on you as a learner to be very explicit. [07:07] around what you want to achieve and how you want to achieve it and giving a constant feedback. [07:11] That's not how people learn. Like when you were in school, you're, [07:14] You didn't give constant feedback to the teacher to get them to adjust their curriculum. The teacher was not asking you questions about how best to structure the course and where to go next. That's what I mean by passive versus active. That makes sense. That makes total sense. I think... [07:28] here's maybe one one thing where i'm sort of slotting oboe into my into my model of the world and maybe one way to put it i'm curious what you what you what you would say is lms as they are are are incredible learning machines and people do in this like very low intent way pick up new things all the time like every single day or at least i do from them and sometimes there are categories of your life where you're like i actually just want to like really learn this thing and i want to

[07:56] get the equivalent of a degree from it or, you know, a diploma or certificate or whatever. And in those cases, you actually need a real teacher that is like thinking about this as a course, not just like a, I'm one-off like reading this book and I like get curious about this particular character. And I'm like going down with going down the rabbit hole, which had you tea on it. Is that, is that, is that sort of what you're, what you're thinking about? [08:17] Yeah, I don't think Oboe is a quick answer platform. There are many, many ways to get your quick answer to things, right? The internet, I mean, [08:24] If you think about... There are many ways to think about LLMs. And I know... [08:29] you know, you've [08:30] written about this and explored a lot of these different perspectives. But at their core, what an LLM is at its core is a [08:39] information compression machine that just takes the massive breadth of the Internet. [08:44] And it gives you the information in a much more personalized, specific, fast way. [08:49] compressed way. That's what it does. It just takes... [08:52] all human knowledge in the form of content on the internet and compresses it down to uh [08:56] to what could be distilled into a back and forth conversation. And so there's an assumption there that the way that people should use LLMs is for... [09:06] quick, succinct information, right? It's not intended to [09:11] be long form. And yet learning, true learning, the thing that people really want to, whatever it might be, that objective that people want to achieve requires, [09:19] a commitment and it requires you to follow through and it requires you to take multiple steps. And anybody who has spent more than a few minutes in an LLM conversation trying to do anything longer than the quick information thing has probably learned, LLMs are not great at that, right? They lose context very quickly. They don't stay focused. It's really easy for them to deviate from the path that you originally set out.

[09:39] And that I think is problematic. That's actually a thing that we talk about at Oboe a lot is, [09:43] How do you allow users to [09:45] engage with [09:47] our platform and deviate and ask questions and make it personalized and yet continue to provide the scaffolding that will always bring them back to the core objective that they set out for themselves. [09:58] That I think is something that LLMs do not. [10:00] do very well. [10:01] I want to show everyone the product. I think it's really cool. And I think that'll give us a lot more, a lot more concrete stuff to talk about. Sounds good. And, yeah, [10:09] is can I pick the course topic? [10:11] Yeah, of course. Live demo. This has never gone wrong. We're doing it live, folks. [10:18] Okay. Okay. The course that I want to take is, I want to take a course in Wittgenstein's philosophical investigations. And I'm sure that it can do this. But the thing that I'm not sure about, and you tell me, is what I, my ideal version of this course is... [10:38] it uses the full text, which is like available for free online. And every unit, [10:46] of the course is just taken it's it's written in aphorisms basically or like paragraphs like subsections and each um each uh unit of the course is like uh [10:58] It takes one of the subsections and then explains it and talks about it, talks about what you need to know to understand it, and then moves on to the next one. And maybe that's a horrible way to learn this book, but that's my vision.

[11:11] Interesting. Two things. First of all, you're going to have to help me spell it. [11:16] but that specific use case i actually think is um [11:22] It'll be interesting to see what it produces. And maybe it's not the best example because one of the things that we... [11:28] built into the engine that powers this is we want things to feel [11:33] I, [11:35] lightweight and achievable from hitting a milestone standpoint. And so one of the things that will probably happen is it'll... [11:42] reduce, it sounds like you envision a pretty substantial scope to this course. It'll by default, try to reduce that to [11:50] the key points because it wants to make you feel like you're making progress along the way and hitting important milestones rather than giving you something that's like an unachievable thousand chapter long i see of course i'm also super happy to like have it be a section of it like the first 30 the first 30 things uh or something the first 30 sections or something like that would that be better let's let's see what it does let's see what it does i don't know let's say um [12:14] All right, how do you spell it? So Vickenschein. [12:18] Wittt. [12:22] G E N [12:25] Stein, Wittgenstein. Okay. [12:29] Uh, Wittgenstein's philosophical investigations. [12:36] Uh, philosophical investigations. I'm going to say give, um... [12:41] Pull out the first...

[12:42] part and give me a [12:46] I would say part is not going to be clear enough. I would say first 30 subsections. [12:51] Okay. [12:52] Thank you. [12:54] and give me the context I'll need to... [13:00] Understand them? Understand each one. Yeah. Okay. Great. Understand them. Interesting. Perfect. Yeah. You know, one of the interesting, well, we'll generate this. [13:10] One of the interesting things that is a... [13:12] tough balancing act that we've had to strike is because we're building this [13:16] The grand vision is you can learn anything with Humbo. [13:21] And building a platform that is able to focus on Wittgenstein while also teaching people all the other things that they want to learn. It's really interesting. It's a really interesting balancing act to figure out how do you empower a platform to... [13:36] to have autonomy and to have the breadth to cover everything while also being opinionated about what it does. Because like I said, a true learning platform has to be built with a learning use case in mind. And so we have to have opinions about [13:48] pedagogical methods and how it is [13:51] we should be presenting information and what is the balance of passive versus active engagement, things like that. [13:57] So, all right. So here's our course. So Oboe creates a course. Let me just, let me just stop you there right there. So this is sick. It's super cool. Um, like for people who are listening, it's a, it's now, it's now a page with a bunch of different sections. There's a headline that says Wittgenstein's philosophical investigations, first 30 subsections. There's an introduction. It has like an explanation. It has a podcast. Um, and then it has a bunch of like subsections. And one of the things I think is cool about how

[14:27] Uh, [14:27] all in one shot, um, all in like a, a very short timeframe. And it looks like you're in parallel generating a lot of different things. And, and, um, [14:38] You are showing me the first thing I want to read so I can get started immediately while you're then also filling in the rest of it, which I think is like really, really smart rather than making me wait. [14:48] Yeah, that's how you're thinking about it, right? Yeah, I mean, look, speed is... [14:54] a critical piece of any AI product, I think. We did not want to be one of those products. And there are other products on the market that [15:00] that do this. We didn't want to be one of those products that requires [15:04] you to submit something and then wait around for a while in order to get started. [15:08] A big part of our... [15:10] value proposition here and our positioning for the product is you can learn anything. Anything that you ever thought was too hard to learn, we could at least get you started and you can feel like it is achievable. It's very hard to do that if [15:21] you give us something and then we immediately violate your trust by taking forever to give you something. [15:28] we're there to be the guide, right? And so we want to take you on that first step as quickly as possible. [15:33] I have a question... [15:35] Like, is there, so it says there are two sources. Did it go and pull the full text? [15:41] In this case, it looks like... [15:45] it may have from these sources. [15:48] And also it's possible that [15:50] It will do that if it feels that it needs to go get an external source. Also, each of the chapters that we have here.

[15:57] um, [15:58] will have different sources. [16:01] And so it's possible that, you know, it may have pulled them into particular ones, but not into the specific one. [16:06] Interesting. Okay. [16:07] Um, you know, this is the beauty of working with AI is like, you don't always know what's happening under the hood. I mean, we've obviously we can dig into it behind the scenes. But when you're looking at the product, a lot of times, even as the person who built this, a lot of times I'm looking at the product and I'm like, wow, it's pretty incredible how much agency. [16:22] uh you know you give it the right guard rails and you give it the right amount of autonomy [16:25] It's able to use that agency to do things that you had never expected. I've never, this is a live experience. [16:30] demo. I've never looked at this example before. I've never seen the content that we're about to see before. So it's always very cool seeing what comes out. [16:38] I love it. So take us through it from the perspective, I mean, and talk about it as because there are people who are going to be watching and there's also people are going to be listening. So, so take us through what's there. [16:47] For sure. Yeah. So a bunch of different things going on here. Multimodality is a big piece of what we believe in. So what that means is you can't just be getting a bunch of text. [16:59] in the way that an LLM would give it to you, you have to be given a variety of information, a variety of different ways that the way that makes is most suitable at any given time. And so a big part of the pipeline that powers this is kind of figuring out what is the right thing to show you. Also, what is the right format to show you at any given time? Now, [17:16] The one thing that is an exception to that is our podcast format, because podcast listening is a very different type of experience. The times during your day when you would want to listen to a podcast are very different than the times that you would want to. [17:26] engage with something that looks like this. And that's why the podcast sits separately.

[17:33] This is a generated podcast. It's a conversation between two people talking about the topic. And one of the things that we've recently added is you can think about this particular chapter as the first episode in a podcast. [17:44] And all the chapters that you see here on the side are... [17:47] additional episodes of this long form podcast. And so the course that you created actually also created a [17:53] podcast with multiple episodes. In this case, it's six episodes because there are six chapters here, each of which kind of build on [17:59] each other and reference things that came before, but you always have the same two hosts. [18:03] If I were to play this, I don't know if this would work in our recording, but you tell me if you can hear this. Have you ever tried to explain a really specific feeling or maybe a dream you had and the words just fail? [18:15] Like the more you talk, the further away you get from the actual thing. [18:18] all the time. [18:19] It's that frustrating moment where you feel like language itself is the barrier. [18:23] And it's funny you bring that up, because that exact problem obsessed a philosopher named Ludwig Wittgenstein for his entire life. [18:30] She pronounces it like "Shteen." Interesting. Yeah, I know. Now I'm unsure. You probably know more than the synthetic voice that we're using there. [18:42] The... [18:44] You know, the experience of consuming this is it's supposed to be rich. It's supposed to be... [18:50] as I said, it's supposed to make use of a variety of different formats. Now, depending on [18:56] what type of content you're creating, you're going to see a different mix of things. If I were, we should look at a STEM example because it'll look completely different than this, right? If I'm looking at something that's very philosophy based, it'll be heavily

[19:06] about the language that you're seeing, a lot less about the visuals, despite the fact that we have a nice photo of, uh, [19:11] Ludwig here. [19:12] We have... [19:16] Thank you. [19:16] You know, you get taken through an experience that, as I mentioned, is meant to feel very piecemeal and achievable, because I think one of the biggest issues, the biggest blockers that prevents people from learning is feeling like. [19:28] it's overwhelming to be able to get to the end state that they want to get to. And so, you know, this case here, I'll show you a different example, just as a point of comparison. If I were to go here and [19:38] So this is a recent course that somebody generated on the platform, Quantum Tunneling Explained. [19:44] very different, obviously. And so as you're [19:48] In this case, you're getting a course about quantum mechanics. [19:50] And you'll see [19:52] Um, [19:54] Thank you. [19:54] examples of visuals and content that's pulled in and content that's generated that looks very different because it's built for the specific use case. As we scale the team, as we scale the product, we want to add support for more and more formats and more different, not only formats, but what we call embedded formats, which means the formats that show up in line at the appropriate moment. [20:14] for you to learn the things you want to learn. [20:16] What would happen if I uploaded a full book? [20:19] We do have size limits on the uploads, however, [20:23] I guess it depends how big your file is. But if you were to do that, if you were to take Fick Finstein's book, for instance, you could upload it. You could drag in any attachment here. And we see people doing that. And we see people using it for-- [20:33] for work for that purpose or help me understand this article or this document or whatever it is. That's what this plus button is right here.

[20:40] And in the case of you uploading a book, I think without giving it direction, [20:45] it would have to infer a lot about what your intent was. [20:48] by uploading that. But if you uploaded the book and then gave it specific intention about [20:52] I want you to analyze this. [20:54] fiction book and do a whole character analysis with plot. [20:58] graphs and things like that, it should be able to do it versus, you know, giving it a completely different set of directions with that uploaded book. [21:05] Okay, cool. Um, that's really interesting. Can we go back to the, to Wittgenstein real quick? Um, I don't want to get too far away from Ludwig. I'll have a attachment anxiety. Um, okay. So if we, if I want to scroll down, so, so it looks like in this, in these chapters, there's like, um, yeah. [21:21] In the chapters that you that you have. It has like introduction to Wittgenstein transition to philosophical investigation, understanding language games, meaning, meaning is used, which are it's going through all the major concepts of the book. But I'm wondering if we go to the analyzing the first 30 sections, I just want to see what it did there. [21:42] Okay, interesting. Let's keep going. So it's basically talking about, it is actually in the beginning of the book. It is actually breaking that down. Let's keep going. [21:52] Scroll to the bottom. I just want to see like where it got to. [21:58] Oh, interesting. And so it looks like it's a more summarized... [22:03] the first 30 sections as opposed to doing what you you know what you suggested yeah yeah that's interesting i i do like i did notice this you have this like quiz thing where um every once in a while there's a format where it like asks you stuff about the whatever it's teaching you to like kind of help you do i guess that's the more like one of the more active things you have tell me about that yeah so um as i mentioned a lot of our uh philosophy here is around what we refer to as embedded formats which is let's put the right thing in at the right time and so when it makes

[22:33] point for me to reinforce the material that I've already covered. [22:37] it'll throw in a quiz or it'll throw in flashcards or, uh, [22:40] you know, a game or different formats as we add support for more and more embeddable formats. And what we're doing here is we're giving this pipeline [22:49] that generates all this content more [22:52] Bye bye. [22:53] a toolkit if you want to think about it like that to [22:56] use just like a teacher has a variety of different tools at their disposal and they're able to use the right one at the right time that they determine is [23:03] is fitting. [23:05] But you'll notice in this case, back to our question about, [23:08] passive versus active. [23:10] It is objective oriented, right? It is intentional in the sense that you could see that the course was built in a way that built up to the thing that you asked it to do. In this case, the end result, I guess, was not as detailed as we were hoping that it would be. [23:23] but it builds up from basics to get you to the end result. [23:26] but on the active versus passive thing. [23:28] it tries to strike the right balance there right so in this case that's interesting it's mostly there for me to be like hey like no no like i you know do more i want a bigger deeper course can i say so we we talk about that a lot currently in the product we don't have the ability to do that well the the way to do that would be to continue to refine your prompts and you can generate as many of these as you want for free that's there's no limit on how many you can generate these however uh one of the things that we talk about a lot users ask for this a lot [23:53] is how can we take content that we've already generated and give the user the ability to continually refine the course so that you could you could say actually here I want you to double down I want you to split this into multiple chapters. Yeah, that's really interesting and then giving more granular control to the user to be able to also say,

[24:08] I want it more... [24:10] tonally in a particular format, right? I'm using this to teach my kid [24:14] you know, fifth grade math or whatever it is. And so we want it to be presented to them in a slightly different way. And given those type of giving those granular controls is going to become an important part of the product as well. Got it. Awesome. How's it going so far? [24:27] like business-wise? It's great. So we launched the product officially in September of 2025. [24:38] And the... [24:40] three months or so following that were all about, um, [24:43] addressing some [24:45] fundamental things that came from our users came from the launch, the realization that we could be much more objective oriented in our content, we could have a much more enriched format, enriched presentation of the material, which is the version that you're seeing right now. And so we recently launched a [25:01] substantial [25:02] uh, [25:04] change in terms of how the product is presented to the user and what they get out of it. As well as some changes like allowing users to create unlimited courses. [25:13] for free. And so that's relatively new. And then the other thing that happened recently was we announced our Series A, which we [25:20] raised after launching the product. And so that was just announced as well. And so we're now entering the phase where we're able to [25:27] grow the team and do a lot more and hopefully build out a lot of these features that you and I are talking about. [25:32] That's awesome. Congrats. What has surprised you so far about like who's using it and what they're using it for and maybe what they're not using it for?

[25:44] I wasn't expecting the extent to which what people put in is... [25:50] objective oriented, as I mentioned. [25:53] And that's informed our roadmap a lot. And so if you look at the... [25:57] the prompts that people put in, more than two-thirds of them are under this category that we refer to as objective-based. [26:04] learning goals. [26:05] And it kind of reinforces... [26:07] It probably should have been obvious to us, but I think actually seeing the real world data changes your perspective on what a product like this is for. [26:14] Most people today struggle, I think, [26:18] with [26:19] this gap that exists in their minds, in my mind, for a lot of things that I want to learn, and I'm sure in yours as well. [26:25] I know what I want to achieve. I just have no idea how to get there. [26:29] I know that I need to understand X. I know that I want to do Y for my job. I know that I want to start learning about this topic that I know I'm theoretically interested in it. [26:39] I have no idea how to even begin. And I have no idea what the steps are to get there. And so I think creating... Learning, to me, is primarily about that. It's about... [26:49] Bye. [26:49] allowing someone to specify what their end objective is. [26:52] and gain them. [26:54] to that path. Now, if you think about the way that most people historically have learned things with online platforms, it's actually the exact opposite of that. They have no control, no agency over where I'm going. [27:03] It's just, I'm generally interested in a topic. I'm going to find some resources for beginners to get me on the first step. And then I have to do a bunch of research to figure out how to get to the second step. [27:13] But you're not personalizing the journey. I mean, even LLMs, to the point that we mentioned earlier,

[27:18] If you were to specify to an LLM what your long-term objective was, and you were to spend more than a few minutes or a long context window trying to talk to it to get you there, it'll very quickly lose context, right? And it'll lose track of what it is that you're trying to achieve. And so ultimately, I think... [27:34] a real learning platform that's, um, [27:38] that's successful at teaching has to be able to simultaneously [27:44] be always oriented towards the objective the user is trying to get to, and also allow for a lot of freedom along the way and a lot of quick wins and a lot of [27:51] keeping the learner motivated so that they continue to go. [27:55] That's interesting. Yeah, I have not found it that, you know, like chat GPT, especially with like now, like let's say GPT 5.2 and Opus 4.5, that it really gets that off track with long context. Opus 4.5, definitely like in longer chats, it'll do this compaction stuff. And it's just like the way they've done that is like stupid, but more or less, like, it seems in my experience that it keeps going, like these chats keep going. [28:25] Was it 03? Yeah, it was 03. They added the ability for it to... [28:30] set reminders. And so what I started to do was I took a couple of things that I wanted to learn about. So one was, I wanted to go through Andre Karpathy's, um, building a language model course on YouTube. Um, and I said like every day, um, [28:45] I want you to like take me through a new section of that course on YouTube. So like get the whole video, figure out what he does. And then like walk me through it step-by-step and stop when I'm,

[28:54] when I don't get something and like, let's just keep working on that until I get it. And it actually worked really well for like a couple of days. And it, it was like, [29:03] actually really good overall because like 200 days in, it was still like beeping me and being like, Hey, like there's this piece of this thing that you haven't done. It was like pretty good. But the problem is that, [29:17] So basically like splitting it up into those little pieces at some point, [29:22] I got to a piece where it was like a little hard, but, [29:27] And then I was like, I don't have time for this right now. And I let a couple of days go by. And then that piece, like looking at it again, three days later, I was like, I don't really have the context anymore for even where we were. [29:39] Um, and, and I have to do now even more work to like build back up to this and that's even less appealing. And so it just like ended up just, they just kept reminding me, Hey, like we're right here at this, like, you know, we're learning about how key value stores and LLMs work or whatever. And, um, uh, and I would just feel guilty about it every morning. And then I just like dropped it basically. So that's, that's been my experience actually, is that, um, yeah. [30:05] One, two, three. [30:07] I often have a like motivation to learn something hard, but that motivation is [30:11] um, [30:12] it passes, like, especially if it's not fully related to like my job or like something. And this is actually kind of, [30:19] related to my job, but like I can get by my job without having done this course. Right. Um,

[30:24] And then, so there's the passive stuff and then there's the like, [30:29] I, we're building up to something and I like sort of lose context of where we are on the path. And then it doesn't adjust to be like, Hey, I saw you haven't responded in three days. Like, let's try to reignite your interest here and like get you caught up in a way that like gets you psyched again in the way that, you know, maybe a friend would or something like that doesn't have that like little, you know, [30:50] that sort of intelligence, which maybe at this point it could do it. I just haven't prompted it for that. But I think those are my two big problems. [30:59] I think that touches on... [31:01] the interesting, uh, [31:03] double-edged sword of [31:06] creating llms and building them to be generalized tools right especially once you get into the agentic stuff which is what you're talking about [31:14] In order to [31:16] have it deliver the value [31:18] You really let this thing run on its own more than... [31:22] an LLM in a regular conversation, right? And so in order to have it deliver value over a long period of time, over multiple sessions, [31:29] You have to be pretty constraining, I think, in terms of what this thing can do. You don't want it to go off the rails, which agents, if you set them off on their own, easily could. And so there's this really interesting balancing act that has to happen under the hood for any of these AI platforms, especially for LLMs, where it's like, as you're building agents... [31:46] How do you set the guardrails, give them some level of autonomy, [31:50] but set it in a way that's flexible enough. And to your point, it's clearly not flexible enough to say,

[31:55] hold on, I need to reassess the entire approach here because Dan's kind of lost track or he's lost interest and I need to come at this from a different angle. [32:04] Clearly, we are in the very early days, I think, of the technology that is able to successfully do that. It's an agent sort of reassessing its own... [32:12] uh, [32:13] its own objective and its own approach to things and allowing [32:16] the system to [32:19] rewrite the rules, redefine the guardrails and move them so that, uh, [32:24] so that it becomes more valuable without requiring you as the user to actually come in and be very explicit with, actually, no, I want you to do this. [32:32] Like my point from earlier, you would never do that with a teacher. [32:35] A teacher would be able to read the room. A teacher would be able to know when their students were confused. A teacher would be able to know, "Hold on a second. It's been a week since we last covered this. I need to reinforce certain material." [32:44] They don't put the onus on the student. [32:47] to come to them and say, "Hey, actually, teacher, you need to do it this particular way." And I think that's where... [32:53] the current landscape breaks down and doesn't fulfill the promise of being the type of learning experience that we all had in school growing up. [33:00] And how do you think about [33:04] Like, [33:05] So, you know, this experience with ChatGPT, it's like, I can sort of set this thing and then I forget it. [33:10] And, but it continues popping up. So it feels like it's now part of my life and it's in the way of my normal routine. Whereas my experience with Oboe, which is using a little bit a couple of days ago and in preparation for this. And honestly, just, I like saw a tweet about it and I was like, I got it. Like, I honestly forgot we were, I knew we were recording at some point, but I was like, this actually sounds sick. I should just try this. And my experience with it.

[33:33] is I asked it to make a course and I can't remember what it was. And I was looking at it and I was like, oh, this is cool. But then I was like, I was on my phone, I was in the airport. And then I was like, okay, like... [33:44] this is cool, there's a lot here. I read a little bit of it and then I just forgot about it. [33:48] basically. Um, and it feels like there's enough material in any course that I could spend probably hours over, over a couple of days on it. But the consumption format feels very like [34:02] ephemeral. [34:04] How do you think about that? [34:07] It's a [34:09] Talking about balancing acts again, it's a tough balance to create content that feels lightweight, but not ephemeral. And I think we are continuing to work on that is how do you make this thing feel like a... [34:20] an asset that has [34:23] longevity to it. [34:24] that you can come back to, but not so much so. [34:27] that it intimidates you and makes you feel like it's heavy. [34:30] I think one of the single biggest issues that exists with any learning platform historically, especially like formal education platforms, is the content feels incredibly overwhelming. [34:40] You're presented with this massive amount of information. It feels heavy. You don't want to even get started. So that I think is an interesting balance. [34:48] Now that we have a model where users can create as many courses as they want for free and put in as many props, I actually think that ephemerality is totally fine. At any given time, we should be able to retain context about what you've asked before. If you want to pick up where you left, you don't need to go into a pre-existing course. We can make...

[35:04] a new one picking up where you left off and you should be able to, [35:07] to prompt Obo to let you do that. I will say, given how early we are, [35:13] We don't yet have the re-engagement hooks that a product like this should have. We don't have a mobile product yet. A native mobile product. It's a web-based product right now. And that's all coming. And obviously, learning requires notifications. Re-engagement learning requires you to learn on your phone and on your desktop and have a native app. [35:33] where we decided to focus our energy with a... [35:35] small team that we have in the early days was on hyper focusing on the utility that this product would deliver. Because if we can nail the value proposition, if we can really find product market fit with [35:45] the use cases that people would come with, then the sky's the limit in terms of where you could take that in terms of re-engagement. What are you using it for? What are you learning with it? [35:55] So I'm a big nerd and I, for years, have been... [36:01] very interested in all of these advanced math and science topics that I never learned in school. I did not major in math and physics, and I wish I had. [36:10] uh end. [36:11] This speaks to the mission of the company also is it took me a very long time. [36:16] to realize that despite the fact that I was fascinated by all these topics, [36:20] that I could actually teach myself. [36:22] And it's becoming increasingly... [36:24] easy for me to teach myself, especially with tools like Oboe. And, uh, [36:30] Yeah, so... [36:31] bunch of physics topics primarily that, uh,

[36:34] that I had never gotten a chance to learn formally in an academic setting. And now I realize, like, I don't need the academic setting, right? I'm able to actually, I'm [36:41] go through and teach myself and have an enjoyable, lightweight experience where I can jump in and jump out as easily as I want to. [36:47] What kind of physics stuff are you thinking about? [36:51] In the case of Oboe, here, let me see what I've made recently. [37:01] I was reading the other day about the history of quantum physics, like how was first discovered in the early experiments. [37:07] you know, how did they discover radiation and stuff? [37:10] More the... [37:13] I forgot how to pronounce it. Have you ever heard of the... [37:18] we're gonna get very uh yeah uh stern gerlock i think it's called do you know it's an excellent follow-up to wittgenstein um i don't know stern gerlock okay so uh we'll get very nerdy here for a second so stern gerlock experiment was the experiment that basically proved that um [37:39] quantum spin, which is the... If you take a particle, any particle, any atom or electron or whatever, it has some inherent spin property where it spins. And there's this challenge of determining [37:52] what direction it actually spins in. [37:54] And so they built this apparatus, these two experimenters, Stern and Gerlach, or Garlach, I don't know how it's pronounced. [38:02] And they built this apparatus that basically

[38:04] determines spin, it measures spin and determines it as either being up or down. And [38:13] That's to be expected. If you throw a particle and you measure the spin of a particle as up or down, then... [38:19] It's one or the other and probabilistically in quantum physics, it kind of expects that, you know, it's 50% of particles would be up, 50% would be down. But there are all these really weird variations on the experiment that they did to determine that quantum states are like totally inherently probabilistic and completely unpredictable. And so to give you an example of this, if you were to take a particle and measure its spin as either being up or down, [38:41] And then you were to measure its spin along a different dimension. So like instead of measuring how much is spinning on the z-axis, you measure it on its x-axis. [38:50] And then you measure it again back on the original axis that you had. So you go from Z to measuring it on X back to Z. [38:56] there is once again a 50% probability that is either up or down on this new axis. So this experiment basically proved, and this had been theorized for a while, but they actually proved it through this experiment, [39:06] like 100 years ago, that the particle somehow almost forgets, [39:11] the state that it's in, it almost forgets which way it's spinning when you measure it the second time. And when you go back to orienting it, [39:19] My computer, I don't know if you saw that. It sounds like fireworks. I guess I did a hand gesture. Mind blown. [39:27] When you go back to measuring it on the original axis that you measured on, it totally disregards the original measurement, which makes absolutely no sense. That's almost as if I were to say, I spin a top.

[39:40] or I spin a ball or something like that. I measure it in one way, I measure it in a different way. And then when I go back to measuring it the first way, it's actually spinning in a different direction 50% of the time. [39:48] Um, [39:50] And the thing that blows my mind learning about this is not only the underlying physics itself is fascinating, but also this happened 100 years ago. Like they were able to build successful experiments to determine that this actually was true. [40:04] something like 100 years ago. I can look up when this experiment was done. 1922. [40:09] This experiment was done more than 100 years ago, and it just boggles my mind. It's like, think about the technology that was available to them at the time and that they were able to figure this out. [40:16] Totally. It's, [40:17] It's really interesting. It makes me feel like we haven't really, I feel like Newtonian physics has made its way through all of culture and society, but like quantum physics has not. [40:26] Um, and we're still stuck in sort of like a Newtonian world in a lot of ways. Yeah. Um, yeah. [40:33] Well, it's the one we experience day to day, right? [40:35] Well, it wasn't before Newton, you know? That's true. We were like, this Newton shit, that's crazy. That's definitely not how it works. [40:46] But I think it is sort of unintuitive. The thing that I've been thinking, because I've been, I guess every nerd just sort of likes quantum mechanics. I've been reading a little bit about quantum mechanics and philosophy.

[41:05] So I'm curious as another non-expert, but someone who I seems to be a lot more, [41:12] smarter and more grounded in it than me. And maybe we can make an oboe course about this to explain it to us live on the show. So you tell me. But I've been wondering about... I think of language models as this... [41:25] interesting discontinuity in the way that we think about how knowledge works and what knowledge is. [41:32] And when we first started trying to build artificial intelligence, we started from like symbolic AI, which is essentially like we're going to reduce intelligence down into... [41:46] set of rules that are human understandable are formal are explicit and we're going to build our way up to something that can learn anything and then we just ran into this like big problem which is it would take more computation than is available in the universe to actually actually do that um and then we flipped to um [42:05] language models, which I think of as being a little bit more of a postmodern technology that learns like countless things. [42:14] implicit, um, [42:17] uh, patterns from, um, uh, like tiny, tiny correlations and like long pieces of text to figure out what comes next. Um, [42:26] And that ended up like working really well in this way that's like, it works, but we don't quite understand it. And we can't reduce it down into that symbolic thing that we, that, that makes it understandable and almost makes it feel a little bit more Newtonian.

[42:40] And the way that we did that is we like embedded, invented, uh, embedding spaces and, um, yeah, [42:48] And there's something about that whole thing that reminds me a little bit of quantum mechanics. And the specific thing that I think of with quantum mechanics is the double slit experiment, where I cannot do the double slit experiment off the top of my head, but more or less, it's very similar to what you just explained, where depending on how you look at a photon, [43:12] it shows up as either a particle or a wave. Um, and, um, [43:18] There's all these different weird variations of it, but... [43:21] effectively, the way that you measure it at the time that you measure it, like determines whether it becomes a particle or a wave. And before that, it's in like some probabilistic in-between state. And what that reminds me of is sort of embedding spaces somehow. It's like, if at different times you measure the same thing and it's different depending on when you measure it and how you measure it, then there's probably some like high, it exists in this high dimensional way that you're [43:51] its dimensions down to something that you can actually measure in the same way that you're sort of you're taking a point in the embedding space and like reducing it down into the next letter. I'm saying this very inelegantly, but I think you probably get what I'm saying. Does that make any sense? Or am I just like being crazy? [44:08] Well, for a few reactions, first of all, I'll tell you, you just reminded me there was a

[44:13] maybe 10 years ago or something, I had always been a huge fan of the double-slit experiment. If anybody listening doesn't know what the double-slit experiment is, I don't think I can think of anything I've ever learned in my life that has blown my mind as much as the fact that this is real. And [44:29] And I remember I had a conversation with somebody about 10 years ago where the double slit experiment came up. [44:34] And we were both just totally nerding out about how fascinating it was. And we had this moment where we were like, to your point earlier around how the world hasn't come around to thinking about this. How is not everybody talking about the double slit experiment? The fact that that actually is a real thing and we don't just constantly talk about it. [44:53] is crazy. [44:55] Because it's probably as incredible a fact of the universe as... [44:59] can be imagined. But [45:01] But you're right, there is another, there's definitely a lot of similarities to the fact that [45:05] There's this other incredible, totally inexplicable thing about the universe, which is that if you take a massive amount of information created by humans, the Internet, [45:14] and you pass it into one of these massive neural networks, it's able to identify patterns that we humans don't even know exist. And it's able to do it. [45:21] in a way, you know, the massive high dimensional space of these embedding spaces that you're talking about, [45:27] means that there are dimensions of... [45:30] human. [45:32] output of what it is that we create that we're completely blind to. And it's not like the AI goes about or the, you know, training one of these embedding spaces. It's not like it goes about and labels these axes for us. And so these, these dimensions exist in this high dimensional space that basically totally define what we're doing.

[45:49] patterns in how it is that we behave that we have no idea how to make sense of. And, um, [45:54] And that's certainly a thing that... [45:56] It is incredibly... [45:59] weird as a human being to try and wrap your head around like philosophically, how is it possible that we as humans were able to create machines that are able to understand our output and predict it so much better than we ever would be? And not only that, but do it in a way that we have absolutely no sense of how they actually work under the hood. Like we know the technical way that they work, but we have no... [46:18] understanding of why they find certain patterns and what those patterns actually represent, because they're just numbers and we can't make sense of them. [46:25] But you did say something that I thought was interesting, which is, you talked about the probabilistic nature of the LLMs. [46:30] It's worth pointing out, LLMs inherently are not probabilistic. Actually, the underlying models that power them are 100% deterministic. We put in the probabilistic variance to try and make it sound more human-like and unpredictable. [46:44] But the truth is, and I don't know if anybody's done this type of experiment, but I would love to see this is like, what does an LLM look like? [46:51] where you reduce the variability of its output [46:54] to zero, which I think it would be the temperature setting, right? You basically reduce the temperature of any LLM down to zero so that the thing that it's determined, it's outputting is 100% mapped to the, uh, [47:04] to the, [47:05] patterns that it had found, right, and into the weights that it's giving to each one of the tokens. [47:09] Um, [47:10] How good or bad would that output be? It probably wouldn't feel very human, but it would weirdly actually be more accurately representative of the output that humans are creating on the Internet. [47:19] That is interesting. I think the reason why that doesn't work currently, there's really good research from Thinking Machines, which is Miramirati's company, where they're looking at why even if you set it to zero, it's actually still not deterministic. And it turns out that it's because...

[47:37] when you add floating point numbers, [47:40] of different precision. [47:42] in different orders. [47:44] you get slightly different results. And those floating point additions are happening in parallel on GPUs. And depending on the order in which the work is batched, [48:01] it will end up changing the end result. And so they've figured out how to fix that so that when you set it to zero, it is actually purely deterministic, but it's like any production LLM [48:13] is not, except for thinking machines, is not actually deterministic even at temperature zero. [48:19] And I guess the reason is, just so I understand it, because it's parallelizing across many GPUs, I guess there's a race condition where you don't know which computation is going to get completed. I think it's something like that. That's interesting. It's definitely about the order of operations of float addition being parallelized in batches on GPUs. You know what? I've never thought about this before, but I actually wonder if the human brain works exactly the same way. [48:49] neural network. [48:50] with. [48:51] gazillions of parameters. [48:54] And so this question arises of take out, [48:56] any consideration of quantum mechanics or anything like that, right? In a purely Newtonian world, if I were to get exactly the same input input into my brain, would it produce the exact same output? And it sounds like based on this research in the real world, because it's noisy and because,

[49:10] different neurons are firing off at different speeds, you actually have the same race conditions happening in your brain. And so it's totally possible that you would have a totally non-deterministic answer or output given the same input in the brain as well. [49:21] I would guess that it's not the same race condition because I don't think we're doing floating point arithmetic. But – [49:30] If you're around old people, [49:33] they tend to repeat themselves. [49:35] Um, and in the same, uh, [49:39] in the same situation, in the same context, they'll say the exact same things and it gets worse and worse as you get older. Um, and I bet there's something, there's something there and it's, I, it's something about, um, [49:50] the flexibility of your neural pathways and which ones get activated and you just end up activating more and more of the same ones instead of new ones. There's something like that I think is going on. [50:00] It's probably a reinforcement. [50:02] thing there too, where as you keep activating the same ones, it reinforces that those are the ones that your brain should be activating, right? And so it ends up getting worse over time. Yeah, exactly. And I think you also, like as a child, you have way more connections than you do as an adult. So you're constantly like pruning connections. And I think that that, [50:19] process maybe i don't know the biology of it but there's something about that process that i think continues and then just you you ossify it a little bit as you get older right [50:27] Anyway, lots to learn. I want to make some MOBOs about all these topics. But I think one thing that you did actually make me think of is... [50:38] Language models know things that we don't know, but that's because we think of knowledge as knowledge.

[50:43] Um, [50:44] something that we need to be able to explicitly talk about. And language models are able to do things that we actually know a lot about implicitly. We just have not been able to articulate the [50:53] in an explicit way, how it works. So we've been able to write for a very long time and we have some idea of, um, like how, how language is formed from linguistics, but that has not enabled us to generate language in the way that language models do. Um, but I do think that there's some broadening of our idea of like what it means to know something from looking at language models and looking at how, um, [51:18] Even if we can't explicitly say how they work, they are actually able to embody a corpus of knowledge that it's not new knowledge because it's all generated from us. So it's just knowledge that exists in us in a different way. [51:30] Wait, say that last part again. Knowledge that exists. It exists in a different way because of the fact that we can't explicitly do that. In a different way. It's intuitive. It's not something that we can talk about, which is... [51:40] actually Wittgenstein's hall point. [51:45] Back to victory. [51:50] Yeah. [51:51] Yeah, look, I think... [51:53] Thank you. [51:54] There is the mind in the internal way that the mind works. And then there's the way that the mind projects out into the real world and the way that we look at it. And I think if anything, LLMs have probably forced us to realize that those two things are massively disconnected, right? Like you can't... [52:09] Just because a mind is conscious and self-aware and aware of its own output doesn't mean that it understands any of the mechanics of how it works under the hood.

[52:16] Uh, [52:19] Yeah, I mean, I... [52:23] I think we're very much scratching the very surface from a... [52:31] physics stamp and the implications on physics and philosophy and things like that of all these questions around like what is the mind and what is consciousness and all that and I think I often think [52:41] try to reflect on like, are we... [52:44] ever going to have some kind of breakthrough that actually answers some kind of these fundamental philosophical questions that you're talking about? [52:51] Um, [52:53] I don't know that I'm convinced that that actually could happen. You know, I think the human mind biologically developed in a way that evolutionarily developed in a way that. [53:02] probably intentionally obscures all this from us. [53:04] and makes it so that we [53:06] We probably can't do the things that LLMs can do when we can't identify the patterns that LLMs can identify. [53:11] Um, [53:12] I don't know. I... [53:13] I think I've convinced myself of that, that there is a limit to how much we can actually know about this, but you might feel differently. [53:19] I think that I've just brought in the definition of knowledge because anything that an LLM can do, a human has done. [53:27] And I think that should count as a form of knowledge. And if we can't explain it. [53:32] That's fair. [53:33] That's fair. [53:34] Although what an LLM has access to in terms of its input and its ability to train on all these different... [53:38] corpuses of any individual person can't but humans have humans collectively can yes correct yeah yeah that's right

[53:46] Yeah. Well, that is, I think, a great way to end it. Nir, thank you so much for coming on the show. If people want to try Oboe or find you on the internet, where can they find you? [53:57] Oboe.com. [53:59] Go create as many courses as you want. Let us know what you think of us feedback. We're always welcome to it. Really appreciate it. Yeah, I'm Neera Zickerman. You can find me with... [54:09] My name is my handle on all the various platforms. So we'd love to... [54:14] hear from everybody. And if you have any product feedback, let us know. [54:18] Awesome. Thanks for coming on. Thank you, Dan. Appreciate it. [54:28] Oh my gosh, folks. You absolutely, positively have to smash that like button and subscribe to AI and I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard. But instead of gold, it's filled with pure, unadulterated knowledge bombs about chat GPT. [54:51] on the edge of your seat. [54:52] craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the spaceship. [55:00] So do yourself a favor, hit like, smash subscribe, and strap in for the ride of your life. [55:05] And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.