Rendered at 13:51:35 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Thanemate 3 minutes ago [-]
I once uploaded a 700 pages book and quizzed chatGPT strictly on its contents. It answered somewhere between 60-90% across 6 quizzes. This was 2 months ago. Those quizzes were fact checking statements, not even problem solving.
I'd be extremely cautious to ask it to have it explain any specialized concept even from a document.
matherial 9 hours ago [-]
This is, without exaggeration, probably the fiftieth blog post or long-form comment about how someone is using an LLM for "complex learning", and I'd just really like to see at least one of these to be accompanied by a statement saying what are the kinds of problems the author can now confidently solve that they couldn't before.
In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.
manfromchina1 4 hours ago [-]
I personally don't get this attitude. For one thing, nobody on earth can give you a 500-word summary of quantum physics in a way that lets you internalize even a tiny bit of it. That's not how you learn anything.
Here's how I do it: I open Baby Rudin (3rd ed.), second chapter, and read the main text - absolute brutality. I unpack almost every sentence with Claude/GPT until I finally get what's going on. No ELI5 nonsense, just examples and counterexamples galore while absorbing the techniques and the way of thinking in analysis/topology. How do I know I've learned the material? By solving every single problem in that chapter. Here's the thing, though: the problems in Rudin can be brutal and decoupled from what's in the text, so if you can handle them, you've definitely mastered the material. No 500-word summary of analysis here.
turzmo 2 hours ago [-]
Baby Rudin is a special textbook: it comes from a time when mathematics textbooks were not at all judged by their pedagogical value, but instead by their aesthetic appeal to mathematicians (a la Bourbaki).
If you actually want to learn analysis, there are a nearly infinite number of friendlier resources (e.g., Understanding Analysis by Abbott).
Learning to unpack difficult text on one’s own is a valuable skill. Research papers often require a similar amount of suffering, and at the frontier of knowledge, despite all the advances we’ve seen, LLMs seem to have absolutely no understanding or intuition. They are much better at things that have been expounded at length by humans before.
I would argue that if you’re going to use an LLM to make Rudin easier to understand, you are not learning how to absorb difficult material, nor are you learning analysis efficiently.
engineer_22 57 minutes ago [-]
Your argument sound remarkably familiar. Humor me:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory.“
"For the correct analogy for the mind is not a vessel that needs filling, but wood that needs igniting - no more - and then it motivates one towards originality and instils the desire for truth. Suppose someone were to go and ask his neighbours for fire and find a substantial blaze there, and just stay there continually warming himself: that is no different from someone who goes to someone else to get some of his rationality, and fails to realize that he ought to ignite his innate flame, his own intellect, but is happy to sit entranced by the lecture, and the words trigger only associative thinking and bring, as it were, only a flush to his cheeks and a glow to his limbs; but he has not dispelled or dispersed, in the warm light of philosophy, the internal dank gloom of his mind."
I didn't read the whole passage, but it seems to be talking about the same thing.
sublinear 3 hours ago [-]
Is that meaningfully different from the study methods of the past? That doesn't require an LLM, and using one risks correctness in exchange for speed. You might not even get that speed if you're stuck in the weeds often enough.
A decade ago, a google search for study guides written by another professor would have been slightly slower. A decade before that, you'd be even slower fumbling through several books. Every single word could at least be trusted. You don't get that from an LLM.
cornholio 3 hours ago [-]
> Is that meaningfully different from the study methods of the past?
The fundamental service a teacher provides is personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area, drastically increasing the speed and quality of learning versus the self-supervised route.
The lack of this closed loop effectively killed the high hopes that were placed in e-learning and MOOCs 15-20 years ago, TV learning in the 1960s and many other failed revolutions, seems every generation has its own version.
It appears to me LLMs have a real potential to close this loop and become the failed educational revolution of our own generation.
alper 24 minutes ago [-]
> personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area,
This has been a huge blocker when I tried to study advanced math myself. Many of the exercise books don't have worked out answers, so often you're either stuck or you have to hunt a variety of sources online for solutions and advice. It kills flow.
Jtarii 3 hours ago [-]
If you have an actual mathematics professor to ask questions to then sure do that, but most people don't have that luxury. Also the error rate of frontier LLMs on textbook highschool/college level mathematics is going to be extremely low.
It is also significantly more engaging and fun.
tcp_handshaker 44 minutes ago [-]
>> the error rate of frontier LLMs on textbook highschool/college level mathematics is going to be extremely low.
The scientific basis for this statement is unclear.
defmacr0 2 hours ago [-]
I certainly would only trust my textbook as the authoritative source, but i can see that in the absence of an expert teacher it's nice to have something that can critique a proof. Imo the fact that it's hard to verify that your own proof is correct is one of the main barriers in self-studying math, especially if one is at a level where one is not completely fluent in applying the various techniques. This also applies to judging answers to open-ended questions in any other field.
winnit 3 hours ago [-]
When studying a mathematics text, it is good practice to question what you are reading, trying to prove things to yourself etc., which IMO makes the problem of trusted sources much less than it is in things which you are not able to verify e.g. historical accounts.
nathell 5 hours ago [-]
I’ve found it helpful to ask LLMs specific questions about your knowledge gaps, or ask for concrete worked examples that illustrate the ideas you’re learning about.
In [0], I ask: „When applying Hidden Markov Models to POS tagging in NLP, what do the latent states and observations usually represent?” I then follow up with some specific questions and requests for walkthrough. You can’t see it from this conversations, but I have Wikipedia and a bunch of other resources open in separate pages, cross-reading, and I follow up with a handwritten toy implementation of a Viterbi-based POS tagger once my mental model crystallizes. This is very different from a 500-word summary of quantum physics, and I still had to put in effort (this is unescapable!), but I found the experience rewarding. Also note that this is relearning of a topic that was part of my uni curriculum but long forgotten.
In [1] and [2], I’m learning Spanish by reading García Lorca’s poems. Here again I’m going through the texts with a dictionary, and augmenting my learning with what a dictionary won’t tell me: given the usage of a word or phrase in this specific poem, is it something that could occur in everyday speech, or is it poetical?
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors.
That's what you'd get if you asked a leading physicist too, so are you saying LLMs have achieved human-level intelligence?
qnleigh 6 hours ago [-]
100% agree. They are not good teachers. Nevertheless I've been using LLMs more and more to learn complex topics, but my learning is always anchored in something else:
- Getting through textbooks and lecture notes. LLMs have gotten very good at answering basic questions on quite advanced material (e.g. representation theory and quantum field theory). By asking a very specific question or even giving the LLM a screenshot, I can get unstuck a lot faster.
- Learning e.g. new python packages. Instead of hunting for examples on Stack Exchange, now I ask an LLM to write a minimal working example and then build off of that. By writing most of the remaining code myself and only using the LLM to answer questions, I've been able to learn new packages significantly faster.
In both cases, the LLM isn't providing the curriculum or guiding what I learn. The textbooks, papers and coding tasks are. But now I can pick these things up much more efficiently.
fy20 9 hours ago [-]
I think it depends on what you want to learn and what your definition of "learnt" is.
The other day I realised I had no idea how DNA and life works. I guess I studied it at high school (25 years ago), but maybe it didn't go into much detail or it just didn't click.
So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
Am I going to become a biologist and study the origins of life from that? Definatley not! But if my kids need help on their biology homework, I now understand the basics of it.
fluoridation 8 hours ago [-]
>So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
If you're only checking your understanding against the one source you used to obtain it, how can you tell whether your understanding coincides with reality (or rather, with general scientific understanding), and not just with the source you read? And I'm not asking just about ChatGPT; the same question could apply to any source. Books are not exempt from containing errors.
Ravus 5 hours ago [-]
Concrete high-profile example: in "Surely You're Joking, Mr. Feynman!", Feynman told of a ball which, in a Brazilian college-level physics book, was described as having a 40% higher acceleration that it would actually have in practice.
Turns out that the author had done a thought experiment but neglected to factor in the rotational inertia.
__alexs 6 hours ago [-]
How can you really say you you've learnt the alphabet if you haven't read the document in which the first use of the letter Y appeared?
fluoridation 6 hours ago [-]
Because you can successfully use it to communicate with other people. That's the grounding truth. If you'd only ever read about it and never actually employed it then yes, you would not be able to make that claim.
__alexs 5 hours ago [-]
That's exactly what OP was doing, or does teaching kids not qualify somehow?
fluoridation 5 hours ago [-]
It's not the same thing. A language is a closed system, it just requires that the participants agree on the rules in order to be successful. Biology is not a language, it's a field of study of phenomena that exist independently of minds. If you read a biology textbook (or an LLM's output about biology) and then relate to someone else what you've read, you and the other person can discuss the topic, and be no closer to ascertain whether what you two understand has any basis in reality.
locknitpicker 2 hours ago [-]
> Biology is not a language, it's a field of study of phenomena that exist independently of minds.
Your personal assertion is quite wrong at a fundamental level. Biology refers to the field or study, not individual specimens. The field is comprised of the understanding that people over time compiled on nature, along with arbitrary frameworks that help people organize and reason about the topic.
Take the concept of species, and species classification. A specimens exists regardless of being classified or not. However, the same specimen can be classified differently depending on the state of the body of knowledge at the time. In some cases you had species being reclassified due to new findings, such as genetic tests.
Once you understand this, you learn that you can't claim that a field of study is anything other than abstractions and partial and incomplete observations compiled form people throughout time.
__alexs 5 hours ago [-]
Biology is in some sense a language, how do we even agree on what the boundaries of biology vs other domains of science are without participants agreeing on the rules?
In any case I think you have an overly narrow definition of learning that we're unlikely to come to terms over.
fluoridation 5 hours ago [-]
>how do we even agree on what the boundaries of biology vs other domains of science are without participants agreeing on the rules?
Those boundaries are completely imaginary and don't exist in reality. In reality there's no biology, there's only elementary particles interacting physically. Whether you agree with someone else to classify a phenomenon as biological or chemical, you're not refining your knowledge of the real world, you're just performing an organizational task.
But, say, how many times the tympanic ear evolved independently is a real phenomenon that can be investigated, and it's something that you're either correct, incorrect, or ignorant about. If I tell you it evolved five times, what more can either of us gleam about the real facts by just discussing this factoid back and forth, if neither of us has access to any additional knowledge or way to put this datum to the test?
elbear 5 hours ago [-]
He's talking about learning that makes you operational. If you can speak with someone, then you've learned language. If you can get a computer to do something, then you've learned some programming.
The equivalent for biology would be to grow a plant or a few plants and animals successfully. That's operational at a certain level, you could also be operational at a lower or higher level.
fluoridation 4 hours ago [-]
No, it doesn't necessarily need to be applicable, it just needs to be testable. The size of the Earth for example may not affect you directly in any way, but it's something you can verify to some degree of precision. But if you just read about it, can you really say that you know it? What if all you believe about the size and shape of the Earth is from what you've asked an LLM? Do you still know it?
EDIT: Perhaps not the best example, because the size and shape of the Earth are data that are repeated often enough that an LLM would be unlikely to quote it grossly incorrectly, but I think my point still comes across.
skydhash 2 hours ago [-]
I’ve been burned by this enough that I no longer say that I’ve learned something if my only interaction with it is explanation from books. You can get snippets of knowledge and a framework of understanding, but true learning only comes with deep interaction of the concepts (practice, simulation, experiments, observations) and not merely reading.
ChrisGreenHeur 6 hours ago [-]
Same question applies if you only check it against your children's biology books
fluoridation 6 hours ago [-]
Uh, yeah. That's what I said. Did you have a point or...?
elAhmo 6 hours ago [-]
For majority of people, especially children, book is an authoritative source. Virtually no one outside of research fields and very excited enthusiasts will check sources in depth to see whether what is shown in the book is true or not.
So, I think the point OP is making is that most people don't really check sources while learning things 'the conventional way'.
fluoridation 6 hours ago [-]
We have to distinguish learning from, let's call it, recreational education. If you're learning a topic, you're not going to be using a single book. Not even schoolchildren do that. Regardless of whether you consider a book an authoritative source, if you read multiple books those books will have to agree with each other, and where they don't you'll experience some confusion until you can resolve the contradiction.
If you're only ingesting information from a single source, be that a book, a teacher, or an LLM, then you haven't really learned, because your knowledge base is unmoored. You can't learn history by studying Tolkien's mythos.
TeMPOraL 6 hours ago [-]
Self-consistency and consistency with your lived experience are good heuristics. Reality is self-consistent, so anything that doesn't add up indicated an error in the source or your understanding of it (or both).
EDIT:
I think it's a kind thing you need to tune yourself into. OTOH, I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it ¯\_(ツ)_/¯.
AgentMatt 5 minutes ago [-]
> I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it
What do you mean by "successful at life" here? Genuine happiness, fulfillment in life? Or in the sense of doing well by what society holds as it's current interpretion of what one should strive for, and otherwise just kinda drifting through life?
Because if it's the latter, I'd say that is to be expected. It's much simpler to put your energy into fulfilling the expectations of whoever is your superior in your current group, mostly get the expected reward, and then just coast. Reflection and experimentation, which is required to get to self-consistent views, takes effort and and the willingness to question existing beliefs, which will also be uncomfortable times.
fluoridation 6 hours ago [-]
How do you do that with topics not directly experimenceable, like quantum mechanics or, as in this case, silicon manufacture?
TeMPOraL 3 hours ago [-]
The very first thing I mention: self-consistency. It's the only thing you have if you don't have any empirical data. It's the only thing anyone has, really. QM scientists reading QM papers and experiment reports of other people, and talking with each other, are still relying on self-consistency to sniff their own (or other people's) mistakes.
RandomLensman 5 hours ago [-]
Not the OP, but I don't think you can. The understanding someone manufacturing silicon has will be not be attainable by reading about it etc. Similar for QM, no replacement for doing (some of) the math yourself, be involved in experiments etc.
snmx999 5 hours ago [-]
Reality is most likely self-consistent, but as we can only experience a tiny part of it it‘s impossible to tell. Also, parts of reality may appear to be contradictory with each other when some pieces are not known.
TeMPOraL 3 hours ago [-]
> Also, parts of reality may appear to be contradictory with each other when some pieces are not known.
Exactly. This tells you where something is off. The problem may be your lack of understanding or wrong understanding, or it may be with the source, or the framing, or you may have hit a genuine lack of data - still, the puzzles don't fit in some area.
And yes, not all self-consistent understanding is correct. But all inconsistent understanding is incorrect. And the more knowledge you gain, the less likely it is that it'll all connect self-consistently, but still be very wrong.
moffkalast 6 hours ago [-]
> consistency with your lived experience
Aka confirmation bias.
We like explanations that fit what we expect, even if they're completely wrong.
TeMPOraL 3 hours ago [-]
I said consistent with your experience, not with your expectations. Lived experience is empirical data.
mariusor 4 hours ago [-]
Can you say in all honesty that a read of the Wikipedia article for DNA would have been less helpful? It's less convenient perhaps, but definitely more authoritative.
gilleain 2 hours ago [-]
Wikipedia is a great _reference_ but not necessarily the best way to learn about a topic. Of course, this depends on the topic, on who has been writing the page, etc
A particularly bad example is higher maths - a wiki pages on a complex mathematics topic often reads like "A gruncheon is a worch in the brashation of plusters" and each of these words is a separate page or topic. Of course, you _can_ in theory 'just' click through all the tree of linked pages to understand a concept ...
For DNA the page (scanning it now) is well laid out, with images (including a spinning Rasmol? image) and lots of detail. However, the detail could be a drag on understanding for some : There are 'nucleotides' and 'nucleosides' and 'nucleobases'? There are non-canonical bases? Supercoiling? Z-DNA? While I know (most) of these things, it is because I've learned about them in other contexts, or by direct instruction.
I'm not saying it is impossible to understand DNA from that page, but it is likely to be harder (for some?) than a more conversational approach to learning.
I just asked Opus to "explain DNA to me in simple language" and the two are not even in the same league in terms of quality.
gilleain 1 hours ago [-]
Fair point - but I wonder how many people that have visited Wikipedia know about simple wiki. I was probably aware of it, but not enough to remember to suggest it as an alternative!
I mean - it is certainly better ... but it is still a lot of stuff. For example:
> Part of an organism's DNA is "non-coding DNA" sequences. They do not code for protein sequences. Some noncoding DNA is transcribed into non-coding RNA molecules, such as transfer RNA, ribosomal RNA, and regulatory RNAs.
Do you _need_ to know about tRNA, rRNA, and operons (?) to understand DNA? The thing about an encyclopedia/wiki entry is that it has to cover the whole topic. This is a strength for reference, as you can scan it and find the bit you need. For learning from scratch, I can see that a conversational approach (with a human or LLM) has advantages where the learner can direct the level of detail and path through the material.
Ultimately, both are worthwhile, but I can also see the strengths/weaknesses of both ways to learn.
Jtarii 3 hours ago [-]
Just reading something is a fundamentally different experience that being actively involved in a conversation about the thing.
sien 2 hours ago [-]
One thing you can do with the LLMs is ask them to make up quick quizzes for you one a subject.
That can help a bit.
lbriner 2 hours ago [-]
Wikipedia is completely non-interactive so of course the experience is different. Do you want to look up something like https://en.wikipedia.org/wiki/Principal_component_analysis and try and understand that from Wikipedia? If you don't understand it, you have to try and click elsewhere. With an LLM, I can be very specific, "I understand x, y, z about PCA but I don't understand why we have to do it? What happens if I don't apply it?" and the LLM most of the time will give very approachable explanations that can be refined further if I still don't get it.
I am studying for a Masters degree in Computer Science with AI and the lecture notes are like Wikipedia sometimes. Incomplete, perhaps assume pre-knowledge that lots of Masters students won't have. All of these I have taken to ChatGPT and got great explanations, diagrams, graphs etc.
bilbo-b-baggins 8 hours ago [-]
Or do you? Cause if you went back and forth with ChatGPT for an hour it definitely hallucinated and lied to you at some point. Maybe consider using something else like Brilliant.org if you want to learn a topic, yanno, so you don’t propagate whatever hallucination from ChatGPT to your kids.
endymion-light 4 hours ago [-]
This would be a valid point maybe 3 years ago, but most chatbots will now query and verify direct sources, especially in research mode.
This is very simple to validate and verify. You could argue it may find false primary sources.
You can condemn models for a variety of other things, but acting as if this is still reality shows a lack of understanding as to modern model capabilities
dns_snek 55 minutes ago [-]
Your comment is phrased as if it somehow refutes their point but it doesn't.
> Cause if you went back and forth with ChatGPT for an hour it definitely hallucinated and lied to you at some point.
If you're asserting that this is not the case today then that's going to be require pretty extraordinary evidence. "Chatbots use Google now" is not evidence that the information they provide is in fact correct.
They don't hallucinate all the time like they used to, no, but I'd be very surprised if the majority of these sorts of conversations were free of major factual errors.
I frequently notice degradation in the model model's ability to remain coherent when it searches for information online. For example I might ask Sonnet 5 "how do I build a shed" and during its search it presumably comes across an article which talks about building a shed out of paper mache, then the model responds with something like "I caution you against your plan to build a shed out of paper mache" -- Wait, what? Who said anything about building it out of paper mache?
abustamam 8 hours ago [-]
These days online courses like Brilliant and such are likely largely LLM generated. Are they actually vetted by experts before publication? Who knows. My money is on no, or at least, not until someone complains.
Its easy enough to prompt ChatGPT for primary sources when doing research to validate any claims its making.
ImHereToVote 7 hours ago [-]
You should actually do this. Ask ChatGPT to check what it said against sources. It does correct itself. The corrections are usually not major. Just mental shortcuts.
Jtarii 3 hours ago [-]
I would trust ChatGPT more than the average elementary school biology teacher.
TaupeRanger 1 hours ago [-]
It's like YouTube "Explainer" personalities, like Hank Green and Adam Neely. They do the kind of "learning theater" that makes you feel like you're learning something when you're actually just providing views and ad revenue. You'll come away from a video feeling like you gained knowledge, but:
1) there's a good chance it was subtly misleading (or just wrong)
2) you probably won't ever use the information in any meaningful way and will likely forget all relevant details in a few days
3) you almost certainly could have spent that time better actually doing or creating something - actually doing real learning and making real progress
tcp_handshaker 47 minutes ago [-]
Plus they will take one hour to explain what could take 2 min, just to keep you on the platform....
cube00 31 minutes ago [-]
The algorithm wanted these "educational" videos to be 10 minutes at first and now it seems to want them to be 20 minutes.
Once I saw that I realised just how padded the content really was and started looking elsewhere.
Other good tells the content is low quality or a bad fit for video are excessive amounts of talking head (should have been a podcast) or stock video clips (it's video, show me something).
ethbr1 21 minutes ago [-]
Listening to YouTube on 2x makes it slightly bearable.
But this was also the joke about TED Talks: everyone came away smugly confident they were part of an amazing tide of progress, ignoring any engineering or scaling challenges.
jupi2142 13 minutes ago [-]
I agree. If it's not followed by a test/exam, then it's entertainment, not learning.
sathish316 1 hours ago [-]
I’ve used the following teach skill by mattpocock to learn Java concurrency concepts in 20 bite-sized hands-on lessons starting from creating a new thread to building a Thread-safe Connection pool. I’m pretty sure I can adapt this to learn Concurrency in any language like Go or Rust.
The point is not that I’m learning Concurrency in a better way using LLMs, it’s that I can apply this style of learning using bite-sized, hands-on, visual explanations, quiz to any topic in the future. The lessons it generates are just code, and you can ask it to type check the examples, validate with recent libraries, use analogies to learn something better.
Try it before thinking it’s just ELI5 or Summarization or assuming it’ll be hallucinating without verifying facts.
embedding-shape 5 hours ago [-]
> I'd just really like to see at least one of these to be accompanied by a statement saying what are the kinds of problems the author can now confidently solve that they couldn't before.
I've spent a lot of time with LLMs for the last two years. Something I've tried, almost for decades, is to learn enough CUDA programming to be productive with it when needed. About 6 months ago, after again banging my head against it for weeks, something finally clicked and I feel like I've overcome the initial step of at least grokking the needed ideas so I know where to go next, and I can actually write + compile + use kernels made for my use cases. I won't claim to understand everything, but I couldn't do what I can today, before I learnt the things I now know.
~2 years ago, because of my very weak math foundation, I basically said "Well, CUDA looks really interesting and really fun, but it's too difficult, lets focus on other things", even after reading some starting resources and stuff. But, by asking countless of dumb questions to LLMs, forcing it to steer me in the right direction, when I'm otherwise just driving on the highway or what not, I finally feel like I have a grasp on something I earlier only dreamed about understanding, and I'm able to be productive with it now.
KeplerBoy 5 hours ago [-]
To be fair the entry barrier got a lot lower over the past ~5 years. Now you can write very good CUDA kernels with just a few lines of python DSL code. Zero cpp boilerplate and zero explicit compiler calls.
Stuff like Triton, nvidia warp (the language), numba, cupy jax/pallas and so many others really paved the way. You can start out really high-level, run a profiler and then dive deep into the bottlenecks.
TL,DR: Keep going, it's a great time to have fun with GPUs.
embedding-shape 5 hours ago [-]
> To be fair the entry barrier got a lot lower over the past ~5 years. Now you can write very good CUDA kernels with just a few lines of python DSL code. Zero cpp boilerplate and zero explicit compiler calls.
Well, yeah, but what I've being doing is learning proper CUDA, not "Python-compiled-to-CUDA" (otherwise it'd take like a just a week to understand enough :P ) and that's looking more or less the same today (although bunch of more complicated stuff piled on top of the fundamentals) as it used to, AFAIK.
With that said, the environment is a lot simpler to setup today at least :)
KeplerBoy 4 hours ago [-]
I wouldn't call one proper CUDA and the other one some dumbed down version. Nvidia really seems to be pushing for these DSLs to be first class within the ecosystem. In some cases probably even more cutting edge than the nvcc frontend, since it's easier to do some experimenting on a new niche package than on the tool everyone relies on.
I believe more and more production code is running kernels which didn't originate from the traditional cuda cpp route.
embedding-shape 8 minutes ago [-]
> I wouldn't call one proper CUDA and the other one some dumbed down version. Nvidia really seems to be pushing for these DSLs to be first class within the ecosystem.
I wouldn't say one is dumbed down either, just different, at least the entrypoints and how you end up using the different solutions.
I'm currently experimenting with cuda-oxide for some new simulations, and managed to keep the entire simulation within just Rust essentially, while going the "traditional" (maybe better term than "proper"?) way I've ended up with a bunch of .cu files and then integrating them (via cudarc usually). Kernels themselves feel the same across both, but the integration clearly makes them different enough that I think it's worth distinguishing them, at least for clarity if nothing else.
If someone else already knew Rust but not C++, wanted to get into CUDA programming, going the cuda-oxide route would probably be easier and more familiar, than cudarc, I'd guess. Personally I'm not sure what route I prefer yet, both (as always?) have tradeoffs.
bonoboTP 3 hours ago [-]
This is not a silver bullet that will make everyone a genius. Not even the best human tutor can do that. But it can be a massive clearer of certain roadblocks for people with no better human help available. Countless people have to resort to learning from teachers who themselves are confused and know the material only shallowly and give confused and wrong explanations that can be very hard to untie for the learner. For most undergrad level things, LLMs have a breadth and depth of knowledge and online search capability that it can make you unstuck on some misunderstanding quite effectively. You still have to work. It's not magic. But the goalpost cannot be to catapult the median person to become Einstein. It's a tool.
Luckily I had my dad available who is a scientist, during my high school and he corrected several fundamental mis-explanations of my teacher that even to my mind logically simply didn't add up. I learned not to relay this back to the teacher of course and sometimes regurgitated the wrong answer in tests. Not everyone is so lucky. This has been my frustration quite often. Textbooks are sometimes wrong, both because the author really doesn't have good expertise on that slice of the topic or out of didactic simplification reasons. Having an LLM that can consult the real grownup literature and give the full story, not the birds-and-bees is quite useful.
yoz-y 3 hours ago [-]
I remember books using simplified models, but being flat out wrong?
bonoboTP 3 hours ago [-]
It is sometimes typos, but even those can be confusing. But okay I'll admit books are usually on the end of simplification, at least for hard sciences. But sometimes they do handle the edges of the authors knowledge and sometimes misconceptions creep in. Famous ones include the reason for airplanes flying or similar "let's relate the hard science to practical reality" and making mistakes there where they think they can just improvise and wing it. Even evolution is sometimes explained very wrong. History and humanities is more murky.
Of course "it's just simplified" is a defense that can be attempted to explain away all mistakes.
matheusmoreira 9 hours ago [-]
> don't really teach you anything
Dunno, I've been learning a lot of Rust in the past few days. Just dove right into a project and asked AI to teach me stuff on a need to know basis. I'm actually getting used to Rust by now.
xxs 7 hours ago [-]
Learning programming languages is quite trivial. Many years ago I counted 16 that I had used professionally, now they are more.
It takes way more time to master and be very comfortable with a language due to its ecosystem, though. Some languages are more likely to click with a person, yet underneath they are all the same (minus the functional languages that form their own group), e.g. some performance issues may require looking at the generate assembly code.
fluoridation 8 hours ago [-]
A programming language is a skill with an accessible source of irrefutable feedback. If the program doesn't work, then you did something wrong. What happens if you try to learn something less concrete or less testable, like quantum mechanics, as the GP suggested?
matheusmoreira 8 hours ago [-]
I agree with you. I'm also using AI to help me learn electronics so I can finally make some real stuff I can hold in my hands. Lots of hard engineering involved so I'm a lot less confident in my ability to spot the AI's own mistakes.
However, saying you can't learn "anything" is just too strong. I'm definitely managing to distill the AI's weights into my own brain.
fluoridation 7 hours ago [-]
It's true in a very real sense, though. Without calibration, how can you tell whether you're learning something real or something fictitious? Imagine trying to learn a language from a single teacher, without ever talking to or corresponding with a native speaker. How could you tell whether you're actually learning the language, or the teacher's unique dialect? Yes, you indeed "learned" something, it's just something that might not bear much resemblance to the real thing.
navigate8310 6 hours ago [-]
This made me realize all those benchmarks which unlike loosely acts as a person being awarded with a certificate after his doctorate is rather worthless, for you never know when it bullshit-ted.
porridgeraisin 2 hours ago [-]
Yep. This is the key. You need some kind of knowledge of what the end result should look like to really learn something from LLMs. Otherwise it's a deep dark forest with no way out.
noisy_boy 9 hours ago [-]
Same. I setup a practice skill that has a curriculum/list of topics, teaching methodology, approach to drills, progress tracking and spaced repetition.
Additionally I asked it to also give me problems relevant in my business domain so that I learn how to directly apply the knowledge in a realistic scenario.
I am getting much more comfortable writing rust than I was barely two weeks ago. More than I was just reading tutorials.
2muchcoffeeman 8 hours ago [-]
AI is a super polarising topic. Even now there are people who are like “it’s all AI slop so it’s all useless” along with the “OMG AI!!! OMG we will have AGI soon!!!” people.
The truth, as always, is in between. There’s loads of people using it for useful things, learning, automation and getting good results. But it’s also wrong enough that you need to deploy it carefully sometimes.
Don’t worry about either group. Keep objective and use AI where it helps and do it yourself where you are better. That’s all.
ixsploit 9 hours ago [-]
I like to use them to generate me tutorial series for technical topics I want to learn or improve. So I start a prompt like this:
> I want you to create a tutorial series about X for me. The prime objective is that I improve in topic X so never provide a solution but guide and teach. (for programming never write code).
First create a question catalog to assess my current level.
Then I would ask it to structure the tutorial challenges in the following way:
- Goal
- Concept
- Instructions
I figured that if I don't need to read any additional material on the topic the LLM is giving me too much information and I need to change the prompt.
Works for me and I used this too learn topics I feel now comfortable with, like nushell, opencyper, elisp, boot loaders etc.
But maybe you don't consider this "complex"
SwtCyber 9 hours ago [-]
I think the key difference here is that you're using the LLM to create exercises, not to replace the learning material
ixsploit 8 hours ago [-]
_Doing_ is probably one of the best ways to learn something.
hlovdal 6 hours ago [-]
"The best way is to learn"
From the hilarious "There are 3 ways to learn. (Knowledge Fight Animated)"
Curious: Why not pick up a book or two on the topic, and use the LLM to help you through it [0]? To me, prompting the LLM without grounding it sounds like a sure shot way to end up learning "pop-sci", as GP puts it, instead of the actual science?
If today's top LLMs are reliable enough (without grounding) to academically learn "complex topics" from, may be I need to adjust my priors. I must say, I do find myself chatting about other topics (without the need for grounding) that I'm trying to "absorb" (not really learn), like Behavioural Psychology & Philosophy.
[0] Products like NotebookLM are built specifically for such usecases.
ixsploit 8 hours ago [-]
I use books to accompany my learning, but I really need something that _forces_ me to think and solve problems in that space on my own.
Books can sometimes give me the illusion of learning something, but then, when needed, I've notice that I haven't really learned it.
LLMs give me structure based on my current skill level. And basically always I accompany this with books, I love reading. It's a nice combination for me.
shuwix 8 hours ago [-]
I would love to see the "depth" of your knowledge in those topics.
You literally proved the OP's point ... thinking you're learning. More like scratching the surface, with lots of invalid data while not being able to recognize what's invalid.
It's like with latest vector of attacks being spamming Github with malware injected in proper looking code in hope of AI to index it.
Then you paste the code because you don't understand it, but you take it as working and only doing what you've asked for.
endymion-light 4 hours ago [-]
This feels like an impossible assessment - yes, a model probably can't give you the education that a advanced/expert book on a topic will, but implying that having a verifiable goal is somehow fake learning feels like an intractable problem.
What level of evidence would be sufficient for you to accept that a model may be able to teach a concept?
I'm happy to take on this challenge with a topic of your choosing, but I don't believe there will be an evidence base that satisfies you that the knowledge is earned or deep enough.
ixsploit 7 hours ago [-]
I really wonder where this assessment is coming from? It's not that I use LLM written code for something (in those exercises at least), in fact I don't let the LLM write code (see my prompt example).
It's about guiding me in _doing_ exercises so I learn and I can evaluate if I learned something if I can apply the learning myself.
akst 3 hours ago [-]
RE: On agents taking up your time
One thing I've begun to notice is that LLMs list of a bunch of interesting stuff and raise all these thing you consider but often sometimes you just want a more focused response, so these scatter responses kind of lead you to being overwhelmed and losing focus on what you really wanted to do. At least I've started noticing this.
Like I'd ask about some statical approach taken in a paper and suddenly i'm being bombard with all these potential pivots and things I really need to consider, I kind of just want to consider 1 thing at a time and come to things once I fixed the immediate issue. Sure I have no doubt these other pieces of information are useful but it's just not the most useful information I need right now.
This is less of a problem with coding agents more so putting learning related questions to an LLM, like is the method covered in this paper, yes no? instead I get an exhaustive but overwhelming and indirect response that contains part of the answer. I just wanted to know if it was worth my time going through the paper but now I'm being bombarded told all this tangential information, which is unclear to me if I need to consider right at this moment, it's really distracting.
Maybe this is something others adapted to but i've resorted speaking past it saying, "this is the question please stay on topic" or literally "one thing at a time please" and then it narrows in, but they really stretch your attention thin if you're not more aggressive with keeping them on topic. The smarter models are better, and if you use max compute it does a better job.
I think they can definitely be helpful for learning, but you got play an active role, you can't just consume what it says like content.
abustamam 8 hours ago [-]
> many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
As a tangent, I think the concept of pop science has wasted so much time of what could be considered brilliant minds. I can't believe how much YouTube people I consider really smart consume under the guise of "learning stuff." And the videos are always designed to be addicting and to entice you to watch other of their stuff, which makes sense, because theyre a business, not a school.
I'm guilty of wasting time on YouTube as much as anyone else (I like watching stand-up routines and Red Bull extreme sports) but I am never under the guise that I'm doing anything productive with my time. Its okay to have fun learning, but I always felt that entertainment and education should be kept separate. You gotta learn something intentionally, not just get it served to you via algorithm.
Note im talking about the educational "shorts" not the 60+ minute deep dives that are basically a college level lecture.
qnleigh 6 hours ago [-]
Well everyone needs some downtime, and then I think it's important to acknowledge that there is a range of quality within the category of "educational entertainment." You can learn a lot of interesting things from 3BlueOneBrown and Veritasium, even though neither compares to working through a textbook.
miki123211 7 hours ago [-]
I really like the term "edutainment" for this.
It's the kind of high-brow entertainment that makes you feel like you learn something.
For me, most "push" things are edutainment, whether that'd be Youtube videos or public-broadcaster television programs. Things you seek out yourself are not.
In a similar vein, there's "newstertainment" (news that makes you feel like it's important to watch, but actually changes nothing tangible about your life).
9dev 6 hours ago [-]
I’m a bit too old for the YouTube generation (can’t stand the fast cuts and permanently agitated voices), but am absolutely guilty of reading a lot of pop-sci books. I feel like they do a better job at teaching stuff for the most part, but probably also make you fall into the trap of believing you actually learnt something…
On the other hand, I’m fine with not being an expert on topics outside of my domain, as long as I retain some basic knowledge and fun party facts. So there’s that.
defmacr0 2 hours ago [-]
There's a spectrum between "barely scientific entertainment" and "dry technical reference". Also, it's not fully a zero-sum tradeoff, great authors have written serious textbooks that are quite entertaining to read, and there are pop-sci books that do a great job at covering advanced material.
sciencejerk 6 hours ago [-]
I can't watch Mark Rober's content because of this. He dilutes his remarkable engineering stunts into ADHD internet memes.
QuadmasterXLII 7 hours ago [-]
I do appreciate the literal college lectures on there. Hard to get the algorithm to surface them but they’re golden for learning. Of course I would be unshocked if the authenticity signals I’m tracking (AB.203 Lecture 3 video title, general hubbub and shuffling of chairs at start of video, university affiliated channel) are all faked by grifters 6 months from now.
abrbhat 3 hours ago [-]
Learning is not about what you put in, it is about what you can take out. It is also not zero-sum, but rather exists on a wide spectrum for any given knowledge domain. A good idea could be to prompt the LLMs to keep quizzing you on what you have read and test on both recall and understanding.
Also learning is not just about truth, it is about curiosity as well. The pop-sci metaphors could actually good for satisfying the curiosity of let's say a 10-year old. What to learn and how to learn is ultimately at the judgement of the learner. The better the judgement, the more the learner can stay closer to the exact scientific details.
yuanBuilds 7 hours ago [-]
In reality, I would just ask LLM to give me a technical explanation of quantum physics, and ask any vocab and equation I do not understand in the response.
sumitkumar 4 hours ago [-]
Agree. Just pay attention to the follow up questions a learner is asking to see the progress. If the follow up is just "continue", "go on", "next" or a non-sequitur then it is smell of a stall. If it is challenging or filling a gap in the answer then it is progress. So production from the learner is the only signal of worth here not the quality of LLM response, the time spent or the ability of the learner to reproduce the facts given by the LLM.
coderatlarge 3 hours ago [-]
prompt: if a family has four boys, would a bayesian conclude that their next child would most likely be a boy?
fable taught me about the beta binomial and large observational studies that came down on both sides of the question about whether per family births are truly binomial. it also told me about countries like the uk and uae that are inching towards national genetic registries that might answer such questions definitively in time. as well as the efforts in Cyprus in this 80s to reduce beta thalassemia through voluntary testing of couples pre marriage.
whazor 6 hours ago [-]
My trick is to let agents make an University Course Curriculum based of books and papers. Then if I don't understand something, I ask AI to simplify the book/paper until I get it.
robertwt7 8 hours ago [-]
> many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
I really enjoyed and learn a lot of things from Karpathy's and Andrew Ng's video. sure many don't really teach you anything, but I could say many others are useful too. Maybe it depends on the way we're prompting as well? it seems useful for some like Terence's message that was shared few weeks ago
I would tend to agree in the case that someone is using LLMs as their primary source for learning. But I've found a lot of use in having a claude project containing the PDF of a textbook I'm working through so that I can ask it to clarify or help me through parts I find confusing. I would definitely say that's greatly accelerated my learning - or at least greatly accelerated the speed at which I integrate information from textbooks.
SwtCyber 9 hours ago [-]
The test I'd like to see: take a problem set or task you couldn't solve beforehand, learn the topic this way, then try to solve it without the LLM in the loop
0x696C6961 4 hours ago [-]
I like to rephrase things in my own words when I read new concepts. The LLM can tell me if my version is totally off.
dalmo3 3 hours ago [-]
Three magic words:
"use Socratic method"
norome 5 hours ago [-]
as ever the real problem is formulating questions, which requires clarity on the next discrete layer of missing comprehension, which requires self-reflection and genuine insight into your own mind.
xandrius 3 hours ago [-]
Have you actually tried? I refreshed a ton of arguments which I had studied or briefly encountered before, so it's less likely to fool me than if I knew 0. And the fact that I can frame a question precisely and as deep as I like is truly unique and incredible.
For some things you still need videos and practice but cmon, I don't get this generalised hate on LLMs, they are based on what us human wrote anyway.
tcp_handshaker 49 minutes ago [-]
To learn from an LLM with all the hallucinations they have...it requires courage.
locknitpicker 2 hours ago [-]
> In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
You wrote a wall of text just to say you struggle with learning when using some media. That's fine, each one of us struggles with different things. However, I hardly think it's fair to extrapolate your personal struggles with learning styles to everyone in such a sweeping approach, or that this is relevant to the topic.
If you want to go back to the basics, LLMs in the very least work as chatbots that you can use to follow the Socratic method to guide your way through your learning journey. If you still struggle with learning when asking questions and getting specific answers to them then it's safe to say LLMs are not a factor.
PreciousH 7 hours ago [-]
[dead]
8bitsout 17 hours ago [-]
I thought LLMs were a great tool for learning new topics - perhaps even complex ones. But overtime, I have had several frustrations with this. First, I get exhausted reading LLM prose. I really don't want to read anything generated by something like Opus 5 at this point. Second, as I dive deeper, I need a way to organize the information in a useful way as I begin to branch out in many different directions. I have tried to use the LLM to fix this by having it generate a web page with diagrams and organized information flow. It's an improvement, but I still run into the issues I described in my first pint - LLM prose is annoyingly dense, and the useful information gets lost in a bunch of noise. You can direct it do something like "use plain English and avoid LLM prose - provide only as much information as necessary to demonstrate the point", but it is once again only a marginal improvement.
And then I begin to think to myself that I should just read a book on the topic written by a trusted source who put a lot of effort into teaching the topic properly and presenting the information in a thoughtful way. So, I am back to books and mostly try to use LLMs to clarify certain questions or ideas I have.
copperx 17 hours ago [-]
It's much better to feed the book to the LLM and ask questions as you read along, instead of asking the LLM to basically write a custom book for you from scratch.
gofreddygo 15 hours ago [-]
I have to agree with this. Completely relying on LLM for all your learning needs is a disaster. But being absolutely against use of LLMs isn't doing you any favors. This is where you don't have a formula but rely on you judgement and evidence of your having learnt something.
For example having an LLM summarize a dense topic and to find books so that you can filter faster and spend time reading those books works way better than having the LLM summarize the books or the topic (or even relying on second hand information). Another one is having the LLM quiz you on your topics of interest. With questions tailored to attack specific areas that you struggle with. Its wonderful at this, nothing I've used comes close to what an LLM can do here.
You define for yourself what your goals are, slowly refining them as you learn more, and use LLM as a tool. This ,I find works best for learning.
intrasight 15 hours ago [-]
Make it your goal to teach a room full of other humans that topic. I guarantee you will know that material cold. I've done lots of technical training in my career and after teaching a class two or three times I find myself to be very competent in the topic.
It's long been the case that the best way to learn something is to teach something.
PsylentKnight 12 hours ago [-]
> It's long been the case that the best way to learn something is to teach something
Which is pretty unfortunate for those that want to learn. I used to enjoy writing documentation at work, it was my favorite part of the job. And it did feel like it benefited me more than it benefited all the people that were (or weren't) reading my documentation. Now I can't really justify spending much time on docmentation when LLM's can do it in a fraction of the time and it's "good enough"
intrasight 12 hours ago [-]
I got started as a software engineer working in the nuclear industry in the 80s. We measured our documentation in inches not pages, and it was all written by hand. And I'll bet you the documentation in the nuclear industry is still written by hand and not by LLMs.
champagnepapi 11 hours ago [-]
I sure hope those docs aren’t written by LLMs for many reasons!
andrekandre 10 hours ago [-]
i wonder if people who use llm's for business critical documentation would have the same opinion, and if so why?
piloto_ciego 9 hours ago [-]
I have used LLMs for safety sensitive tech writing and so have people I know. It’s pretty good at it, and you usually have a 95% ready to go product at the end of it. The last 5% can pretty easily be filled in by technical experts with way less time spent battling the type writer.
Is it perfect? No. But it suffices most of the time in a pinch.
Can’t speak to the nuke industry, but it’s pretty good at aviation related things.
intrasight 2 hours ago [-]
A good compromise would be to have the LLM write the outline. That also helps get over the activation energy of starting.
Then elaborate in small chunks so the human domain expert can remain focused during review.
unethical_ban 11 hours ago [-]
There is still value in experts distilling knowledge and crafting it to the audience. I'm a consultant in cybersecurity and someone asked me "give me a best practice framework for good policy hygiene". I'm sure an LLM could spit out some tips, but I've been on the industry 15 years and can write in 5 pages what an LLM wouldn't conceive of in that space.
qnleigh 6 hours ago [-]
> Another one is having the LLM quiz you on your topics of interest
This is a great idea. I'm going to try it.
SchemaLoad 14 hours ago [-]
Even with the latest models today, the hallucination rate is absurdly high on anything deeper than surface level knowledge or something that can be directly scraped from reddit.
And you notice when it's a topic you know well or something like software where you can immediately tell the options it's giving you don't exist on the page. Leading to the amusing statement "LLMs are bad at what I do but great at everything else".
14 hours ago [-]
unclebucknasty 14 hours ago [-]
I've (elsewhere) written about this diminishing return effect on LLM utility in relation to increasing expertise.
The question: what's the net positive gain of turning people who know nothing in a given field into sub-novices, while weighing actual experts down with work slop and marginal returns?
And I wonder what the true cost is of arming so many novices with that level of dangerous knowledge.
DrewADesign 13 hours ago [-]
The fact that some people get genuine value from LLMs when learning doesn’t contradict the fact that they’re Dunning-Krueger “expertise” generators. The fact that the person learning from them is in charge of ensuring they aren’t full of shit, which they frequently are, is an inescapable flaw in this process. I honestly think that reduces the value of these things to just above what you can find out with a search engine with most topics. Hey, great. An improvement is an improvement right? Is it an improvement worth trillions of dollars and screwing over writers and artists worldwide? Fuck no.
unclebucknasty 12 hours ago [-]
Sounds about right.
Tangentially, but related: I'm old enough to remember when the spirit of your comment was pervasive on HN.
DrewADesign 10 hours ago [-]
It wasn’t even all that long ago. I noticed around the time ChatGPT 3.5 was released I saw a significant change in tone regarding LLMs and diffusion models. I think that’s when some serious astroturfing started.
I’m old enough that I worked my first IT summer job the same year slashdot was founded. I’ve seen a lot of tech tribalism form and dissipate, and this one didn’t feel organic. My gut says a lot of the us-vs-them tension originated in a deliberate campaign to cast AI boosters as the tech industry in-crowd, and ‘other’ the people not on-board. Who knows.
unclebucknasty 9 hours ago [-]
1000% astroturfing—and across social media. Too many posts of the same quality at the same time(s). These things ran in cycles, spinning up and dissipating just as suddenly.
E.g. the entire framing to combat complaints about shortcomings was, "It's not the tech. It's you. You're just not doing it right. Wrong setup, wrong workflow, add this to your .MD, use loops, etc". Every complaint was immediately met with this same treatment by a swarm of vague bro-bots that materialized from the ether. The core message? Always the human's fault.
And, don't get me started on the waves of newly minted expert AI creators, making recommendations without showing a single example of what they'd supposedly built.
I'm sure some bandwagon organic creators tried to cash in on the genre, but encouraging that was also part of the point.
champagnepapi 11 hours ago [-]
I think the true cost will be some catastrophic failures.
Just hoping folks don’t get hurt due to people not understanding what they’re doing with these things but believing they’re competent.
DrewADesign 10 hours ago [-]
If that is the case, I seriously doubt the blame would ever touch the people that oversold this shit.
14 hours ago [-]
gb2d_hn 17 hours ago [-]
I came across the socratic method recently, and have used it to learn a couple of topics that I was having trouble getting to stick. There are some SKILL.md's available for it. It works for concepts as opposed to facts, and causes the model to guide you to answers through your own reasoning, which is both much more engaging than reading a wall of LLM text and helps the information stick.
voidhorse 14 hours ago [-]
The "Socratic Method" (aka maieutic) skills annoy me, precisely because when you read them they are the kind of low-effort, low-expertise crap someone who over relies on AI would naively come up with when tasked with the problem of coming up with skills for learning. "Hey the Platonic dialogues are pretty cool and smart, let's do that".
The body of literature on learning theory, and beyond that on specific types of learning and specific mediums such as learning from text is so rich there are way more useful models to draw from. Believe it or not, prellm, researchers in the textual learning field had already demonstrated you can achieve performance equal or better than novice tutors using pretty basic computer aids that follow specific hint/pump interaction structures. Guiding an LLM to use these findings has evidence backing it and is way better than telling it "i guess be like socrates". The problem is, to realize there might be richer more effective and highly researched ways of tackling the problem beyond the first fart of a thought you had one afternoon requires the deep respect for expertise and specialization that precisely basically everyone in the AI space right now fundamentally lacks.
lambda 11 hours ago [-]
Can you provide some references to the research you're referring to? Sounds interesting but you haven't really provided enough information to find it.
pasc1878 6 hours ago [-]
That is the point – you can't just provide the references in a short post here.
You need much more time and guidance.
gf000 3 hours ago [-]
Ironically, LLMs are pretty good at finding stuff like that from vague descriptions. With that, I think the parent commenter meant Art Graesser's AutoTutor work based on the terminology used (hint/pump).
TLDR: Actual human tutoring sessions were recorded and analyzed and Socratic questioning was barely used at all. Instead the following pattern was observed:
Pump — "Uh huh?" "What else?" Costs nothing, so try it first.
Hint — points at the region of the answer. "What about the pumpkin's motion sideways?"
Prompt — fishes for one specific word, with the sentence frame supplied. "The pumpkin keeps moving forward at the same ___?"
Assertion — just says it. "It keeps the runner's horizontal velocity."
Hardwired8976 17 hours ago [-]
Can you link the Skills.md?
copperx 8 hours ago [-]
> I came across the socratic method recently
This statement would out you as someone who didn't attend an elite school.
hirako2000 17 hours ago [-]
I was so annoyed I made a Socratic wrapper based on predefined curriculum:
Trying to diagrams/animations didn't yield good results even with frontier models. But pure text, any model does a decent job.
hirako2000 15 hours ago [-]
I didn't expect this to be hit by hundreds of request per minute.
So it may be very slow or become unavailable, back end can't handle that, no caching whatsoever.
logicallee 15 hours ago [-]
what a time to be alive! "if you're having trouble understanding what your robot tutor is trying to teach you, you can ask it to guide you to the concepts using your own reasoning. This is both much more engaging than reading a wall of the robot's text and helps the information stick."
giancarlostoro 12 hours ago [-]
Whats funny about LLMs is they are trained from books, but they are also trained to not output books, so how much of an LLM skews its output because a perfectly normal sentence could be a quote in like 300 different books?
trueno 16 hours ago [-]
i threw the entire sanderson cosmere into a RAG graph sorta deal just to see how it would do if i questioned an mcp server for it about a universe i know decently well. it was actually astoundingly good. was able to find easter eggs acrossed different books and answer dumb questions like "why is kaladin emo"
malfist 14 hours ago [-]
If you don't know why kaladin is emo, did you truly read the books lol. Every character has to deal with the stresses of war and most don't come equipped with good mental health to begin with, they're just normal people
trueno 4 hours ago [-]
i was more or less testing to see how complete of an answer it could come up by scanning the whole semantic graph so i threw questions at it that spanned multiple books
etherlord 16 hours ago [-]
Sanderson is 100% in the training data
trueno 4 hours ago [-]
damn it good point
peab 11 hours ago [-]
I also agree with this. LLMs are a great companion when reading a book to clarify things and dive into specific topics.
I'd imagine an application that uses LLMs will be created that better manages learning. It's just not clear what that UX is yet- it's obviously not just a chatbot
arbitrary_name 15 hours ago [-]
dumb question, but what is the best way to feed the book to the llm?
i run into context window limits, or practical limitations of digitizing the book
copperx 14 hours ago [-]
You need a non DRMd copy of the book. You don't have to feed it all at once, although with a 1M context limit it is doable. A few chapters at a time is enough, in my experience. An easy alternative is using NotebookLM (now Gemini Notebook), and that has worked brilliantly for me, but I haven't tested it for technical topics (for that I like the LLM to create graphs and e.g. interact with Mathematica, so I haven't tried it).
12 hours ago [-]
r17n 14 hours ago [-]
Hoping you get answer to this. I have the same question.
broast 11 hours ago [-]
I prefer to have the LLM ask me the questions
schainks 15 hours ago [-]
This. Ask it to quiz you if you are feeling it.
calf 16 hours ago [-]
LLM is still too verbose, a real person Socratic conversation can interact a couple sentences at a time, not spew 1-3 windowfuls of low density bullet points.
I even wonder if this behavior is due to next-token prediction architectures, somehow.
knollimar 13 hours ago [-]
I don't understand some previous complaints. It's dense and verbose seem at ends to me.
I know you probably don't consider it dense but wondering if someone can shed insight.
I find them like empty calories, like programming youtube tutorials. They maximize for feeling learnt instead of steady progress
hirako2000 14 hours ago [-]
There are ways to ground an LLM to be concise, and Socratic (method of inquiry, back and forth dialogue)
voidhorse 14 hours ago [-]
This. This is exactly how I use them and I have had no issues so far. I read the book myself, then I point the LLM at it to ask questions about notions I might be struggling with.
15 hours ago [-]
sojournerc 12 hours ago [-]
I'm sorry, but why not just read the fucking book if you're interested?
sunprema 16 hours ago [-]
[dead]
abound 16 hours ago [-]
This is the exact opposite of what the parent comments are talking about.
sunprema 13 hours ago [-]
I agree, I too like to read authored books. But there are certain topics, especially the new ones doesn't have good books yet. My post was to show that you can create books based on your interest, in a way that you like w.r.t to the content/tone/layout etc. It's a new idea made possible by LLM's. It might become the norm in few years I believe, where each one will be having a personlized library of books that they curate. So the point was to persuade the parent commentator that some books are better created this way.
rDr4g0n 12 hours ago [-]
LLMs can't "read the room" and infer how much context the audience already has, so they try include everything.
human conceptual thinking is very much a multi-dimensional graph, which relies on light "approximate" concepts that are "good enough". LLM AR token generation is extremely one dimensional and doesnt care about the "weight" of the concept behind a token.
LLMs hold billions of parameters in "mind" at once. humans hold like four "concepts".
This is the essential mismatch and the primary reason LLM conversation can be so painful and exhausting.
Explaining this and limiting "concepts" to four at a time tops is one of the very few AGENTS.md / system prompts I always use, and it has proven invaluable time and again.
Thinking traces show how effective this is at forcing the LLM to simplify its thinking.
[edit] Also, myself and nearly all of my peers are struggling to choke down the flaws of LLM tooling along with the benefits. the speed at which LLM adoption is being forced, without truly crafting them into quality tools first, is not ok, and not normal.
LLMs have stirred an inhumane hunger and fear. the tech is fine, but the way tech companies (creators and consumers) are behaving should be deeply questioned.
it's NOT normal. it's not ok.
PaulStatezny 9 hours ago [-]
Would you be willing to provide an example (even a contrived one) of how this "four at a time" prompt changes the LLM's behavior?
I just want to understand more.
Also, would you be willing to share the actual text of it that you put in AGENTS.md?
bumby 11 hours ago [-]
I’m curious where you get the estimate that humans hold “like four” parameters in their mind at once?
If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
rDr4g0n 11 hours ago [-]
updated the comment. i meant four "concepts". i dont reason about my own thinking in terms of parameters.
rl3 17 hours ago [-]
>I really don't want to read anything generated by something like Opus 5 at this point.
Personally, I find that its generated prose tends to have an undue weight to it, almost as if every topic I ask about somehow bears a heavy burden, or is otherwise load-bearing, to use its parlance.
Quite puzzling, really.
roywiggins 17 hours ago [-]
It has a sort of metronomic quality. It never slows down or speeds up or modulates its tone. It plods forward at a relentless pace and never has a light touch with anything.
I think this is one reason why LLM text is pretty exhausting to read for long stretches.
rl3 16 hours ago [-]
>It has a sort of metronomic quality. It never slows down or speeds up or modulates its tone.
It's possible that this quality you describe stems from the extensive training corpora utilized by the major AI labs. These almost certainly include work from the esteemed economist Jacob Silj:
I have a personal theory: LLMs are *fundamentally* handicapped at perceiving what's going on in the mind of the human (this can't be "innovated away") and that's at the root of what makes them suck at conversation.
Next time you're chatting with someone, notice how much understanding is shared without anything being said. E.g. the other person might share something deeply disappointing, and they can tell without you even saying anything whether you get what they're going through. This unspoken-yet-communicated information guides the conversation. Or as another example: humans can read the room -- you walk into a room and immediately adjust your demeanor based on what you see and sense.
LLMs are totally blind to things like this, and this adds an inescapable awkwardness to interacting with them. I don't believe they'll ever grow out of this. Which thankfully implies more long term demand for humans instead of robots. :)
cromka 14 hours ago [-]
Very well observed. I found one more thing: they fail to consider what a 3rd person might understand from your conversation, so when you ask it to dump stuff into a Documentation, they keep making references to facts you had previously discussed or to the train of thought, completely irrelevant to bystander.
PaulStatezny 13 hours ago [-]
100% -- this is the worst. Referencing all sorts of "words with made up contextual/analogous meanings" based on the conversation...outside of the conversation.
Does anyone have a read on if this is primarily a Claude issue, or if all LLMs do this?
cheesecompiler 13 hours ago [-]
Tech guy discovers conversations with humans.
knollimar 13 hours ago [-]
Even over phone calls you get a sense so unless it's in timing it isnt demeanor either
wannabe44 11 hours ago [-]
No, they're just trained to impress the C-suite motherfuggers with dense vocab.
khuston 13 hours ago [-]
I never used office hours as a student, which I later regretted because it made me work longer and harder to perhaps achieve somewhat better understanding in some classes, but also I dropped every proof-based math course I ever took. Overall I think my education would have been stronger by attending office hours.
I view LLMs in education similarly to office hours. Some people abuse it to get homework answers without grappling with the material, but the optimal amount is not zero.
LLM certainly not a replacement for a book, where you get someone’s extended personal approach to a topic, thoughtfully organized, reviewed and edited, often times actual courses taught based on it, with answers checked and errata available online.
relativeadv 13 hours ago [-]
I completely agree. I have vibe coded what i would consider to be some pretty weird things in the name of learning facilitation.
Perhaps the best example has been a native macOS app that is a completely custom text editor with built-in debugger, lsp support, fuzzy finder, etc stuff you'd expect. Inside the same app is a library of books i can read within the app completely formatted and for every chapter/section of each book that is a quiz to take (LLM generated of course), a "recitation" tab where i am asked a question and say outloud my response to the AI to evaluate me on and then finally practice problems to do within the custom text editor (these are usually programming books). The reader also has ai re-write built in.
As neat as this is, and i worked through K&R like this, i have ultimately fallen back on "just read the damn book and go to the AI when you've got questions."
avilay 13 hours ago [-]
I have had very similar experience! I wanted to learn Probablistic ML, checked out a couple of MOOCs, but didn't find any that were at my level - some were too advanced, some too beginner level. Claude was unable to one-shot a course, so I am now asking it to generate it module by module. But even here, it is not doing a very good job. I muddle through the concepts that it has written, do a whole bunch of back-and-forth, which tbh is exhausting, and then rewrite everything in my words so it actually makes sense to another human being.
> I get exhausted reading LLM prose
So much this! If I see one more sentence with the words "genuinely" juxtaposed with "load bearing" my head is going to explode!
i’m personally deriving a huge amount of value from the custom materials fable is assembling for me. for example i asked it to write a focused expository math paper on reed solomon to accompany an implementation module that it wrote for me. it’s remarkably useful to steer it to create graphs and diagrams of exactly how you like the material presented. or the bibliography researched and cross-linked with the body or the order you want your questions addressed.
it also researched vision correcting displays for me and i can finally put that idea to bed - i was never really going to pick up an optometry textbook tbh. plus it was able to pull together a bunch of geometric and physical context about light and the eye plugging exactly my personal knowledge gaps.
in general i suspect these materials might not be that interesting to others because they are so custom to my learning style and personal needs and preferences.
these are usually not one shot documents but rather many prompts deep before i get something I’m willing to sit down and read or study. but dramatically quicker than assembling it myself from primary sources. i wouldn’t say it matches master expositors but then they’re not available to write on any topic i happen to need right now.
plus I’ll just have a live voice discussion with the system when i go for a walk and there are still things bothering me on a topic. it takes a little patience but if i’m in the mood it’s amazing.
i generally find that it can help track down specific references if i suspect hallucinations. but especially on factual topics my experience so far has been extremely encouraging.
avilay 10 hours ago [-]
Good point. Same with me. Despite it being very tiresome, all the back and forth that I do with it really deepens my understanding of the topic. I have been on Opus so far, let me try Fable and see if it gets better. I haven’t tried voice either. Next time I go for a walk I’ll try that!
coderatlarge 3 hours ago [-]
curious to hear about your findings!
mikenew 16 hours ago [-]
I've decided that the main thing I'm building is my own mental model. You can take notes, create docs, put graphs and websites together, but unless I'm just trying to generate some reference material the only real objective is to develop the understanding and intuitions inside my own brain.
So I have the LLM offer a very short explanation of something, and from there's it's just me asking questions. Anything that feels fuzzy or not fully internalized is something I poke at until I'm satisfied.
It really has helped me develop a sensitivity to what I understand vs what I don't, and the ability to drill into any part of it is amazing.
hank1931 15 hours ago [-]
I do this too. I use Claude. I picked a voice I like. I go on a three mile walk. I will ask it questions about a topic that I want to learn about. If it starts telling me more than I want to hear right then, I will say "stop". It doesn't get offended. I then ask it something else. I find this very effective. I control it so it only explains to me what I want explained. If what it says sparks questions on a related topic I jump to a brand new topic. No personal tutor could keep up with this or adjust to exactly how I want to be addressed like Claude does. I'm very excited about the progress I'm making mastering new topics.
And yes, it is not that it is just presenting the facts. By me taking control of the direction the questions and answers go, I can flesh out my mental model. I won't retain every little thing it tells me. But I am much farther ahead than before.
le-mark 13 hours ago [-]
Can you say what topics you’re using this method to learn? I think it would be more effective for some than others.
wanderingmind 13 hours ago [-]
Good to know other people who get migrane reading LLMs dense prose. I started reading books again recently, since everything online is polluted by LLM prose. What i realise, is that a human author, especially a teacher understands the learning pathways of new learners, they motivate the learning, and start from simplest concepts (a spherical cow), and then building all the complexities. This helps us to emphasize on most important concepts, while throwing away unnecessary complexities. While reading LLM prose is like reading a research article, that is written to an expert in the area, that talks about bleeding edge, with full of jargons, caveats, that just is not conducive to the learning process for a new learner.
tkel 12 hours ago [-]
Also they tend to assemble complex jargon in obtuse or meaningless ways, which makes reading and parsing and understanding much more difficult. Tends to reveal that LLMs fundamentally do not have "understanding", just likely word generation
resonious 13 hours ago [-]
To me, it's just Claude. The other models have their quirks but nothing is quite like Claude.
But even with Claude, it's it's really the prose getting in the way you can install the caveman plugin or tell it to use that "standard technical English" thing.
PaulStatezny 10 hours ago [-]
Can you elaborate more on juxtaposing Claude's terrible prose with other LLMs?
Any more detail you can share? Do the others feel more "human"? Are there any that are particularly digestible/human-friendly?
I've been wondering for a while if this is just Claude because I mostly use Claude, so this is very telling.
resonious 10 hours ago [-]
I wish I had something more methodical I could show. It's all subjective, but GLM-5.2 feels more human to me. Even GPT-5.6 Sol tends to be easier on the eyes for me (though the stereotype of it overengineering and no common sense are still true).
I tried using a new agent service recently and could tell immediately that it's powered by Claude due to the way it writes.
andy99 16 hours ago [-]
The sycophancy is also a concern, it’s not really an impartial teacher, all its training is to suck up and maximize engagement rather than learning. The incentives are wrong.
cromka 14 hours ago [-]
> all its training is to suck up and maximize engagement rather than learning
That's speculative, isn't it
adastra22 14 hours ago [-]
No, that's a direct consequence (intended or not) of how RLHF works.
thisisit 16 hours ago [-]
I have been using LLMs to help me turn my journals into interconnected notes and sometimes it is so confusing to read the notes that it doesn't resemble any human would write. Its like the models are getting stronger while also losing its touch to write human sounding sentences on complex topics.
cout 15 hours ago [-]
I use the LLM to point me at relevant books and papers. But there is still a trust problem: I am trusting the LLM to point me at reliable, trustworthy sources.
I'm not sure I'm better off with humans though -- I'm not qualified to judge whether a source is a proper authority, not an I qualified to judge whether someone knows enough to point me to a reliable source.
It seems this is a fundamental epistemological problem to which there may never be an answer.
QwenGlazer9000 13 hours ago [-]
I have the exact same experience, so reassuring to know I'm not the only person who feels this way.
I will say, opus 5 is an egregiously bad case of this, but other LLMs have this too, just less bad.
You can either install that skill or put the Rules section directly in your Global CLAUDE.md for Claude or Personalization setting for Codex and it should cut down the output verbosity by quite a fair bit.
eru 12 hours ago [-]
> First, I get exhausted reading LLM prose. I really don't want to read anything generated by something like Opus 5 at this point.
Agreed.
I find Opus 5, and even Fable, to be overly wordy in eg PR descriptions and code comments.
However, I suspect that's more to do with what they are trained to do by default than LLMs in general. I have a little setup where I tell Claude to work together with Codex to tighten up prose and comments, and for me that produces much more palatable text that needs less human editing afterwards.
tiberriver256 3 hours ago [-]
You should read this article sir! Just scroll to the top of all these comments and check the title!
xfq 12 hours ago [-]
I think one problem is that books are not customizable, and many books are aimed at people with some certain knowledge. With LLMs, you can tell it what your knowledge level is and ask it to customize the answer for you. This is difficult to achieve with books.
supriyo-biswas 12 hours ago [-]
> I get exhausted reading LLM prose.
While my advice is specific to learning about codebases, the way I do it is to have it generate mock data and put it in the local development environment, and give me some exploratory commands, and then ask away. It's a machine after all, so I don't have to read its preceding prose to understand whether it did tell me something, it can just repeat it however many times I ask it, and the hands on commands etc. give me something to actually try and implement.
planckscnst 13 hours ago [-]
You can make something like the link below. I'd say it's a more than marginal improvement. It's still tiring, but it's much better according to my taste. I imagine everyone would have their own version of this for their own preferences.
It's a loop that uses adversarial review to check several dimensions of the writing:
> I really don't want to read anything generated by something like Opus 5 at this point.
Recently switched to OpenAI and I've gotta say Sol is so much better at writing than Claude. Opus has a distinctive sentence structure and Fable somehow manages to be even more obtuse. The personality of these models really does come through...
umpalumpaaa 10 hours ago [-]
Claude code has a „fork“ feature where you can fork an existing conversation and keep talking in the fork and then you can go back to the original of the fork. You can fork as many times as you want and let LLMs write to a markdown file to keep important facts and learnings - also good for agents to do research without expanding the context window
StableAlkyne 12 hours ago [-]
> LLM prose is annoyingly dense, and the useful information gets lost in a bunch of noise
This is my biggest gripe with reading AI-generated text as well (ignoring the meta issue of whether it's worth taking the time to read something that an author didn't think was worth the time to write). It's gotten to the point that weird AI-style analogies just take me completely out of the text and kill my interest.
I think there's an underrated difference between information generation and pedagogy here. LLMs are very good at producing more explanation, yet "more explanation" is often exactly what you don't need when learning something difficult
ethin 16 hours ago [-]
Even if you tell the LLM not to use LLM pros they (still) do it. If you feed the Wikipedia article on signs of AI writing and tell them to use none of those signs they will also (still) do it. I have tried (many times) to get an LLM to explain a concept to me, or a process, or an algorithm or what have you, and every time they cannot help themselves. Either they use LLM pros, or they get so verbose that it all just becomes noise and I spend more time filtering out unnecessary jargon than I do reading let alone learning anything.
cromka 14 hours ago [-]
I ask them to use Simple English and a jargon of the domain. This seems to work best for me.
big_paps 7 hours ago [-]
For Gemini 3 this seems bit to be the case. If you use gems you get completely different personalities - so different that its almost scary. You can create gems which are really insulting, gaslighting or seemingly of a specific profession
Arn_Thor 6 hours ago [-]
> generate a web page with diagrams and organized information flow.
Sounds like you'd be just as well off link-surfing Wikipedia?
KronisLV 15 hours ago [-]
> First, I get exhausted reading LLM prose. I really don't want to read anything generated by something like Opus 5 at this point.
I’ve found the tone of Kimi K3 to be less obnoxious. Unfortunately it doesn’t wholly solve the issue, I don’t think any LLMs out there have a truly pleasant writing style, but at least not every assumption is “load bearing”.
frogperson 14 hours ago [-]
I've found it helpfull to ask the LLM to generate a sylybus for the topic. treat the sylybus as a design doc for a price of software. i find they do much better when they have subtasks to focus on. they can do big picture and small picture, but they can't do both at the same time.
hudn33 13 hours ago [-]
Same. I usually just read the book along and ask questions on a specific part, rather than trying to get the LLM to produce an entire study guide for me. It seems to work better as a Q&A than a "teach me" advisor
c_moscardi 17 hours ago [-]
Yeah, I am currently trying to work with Opus 5 to refresh myself on deep learning fundamentals, and... it's a mixed bag. I'm glad I already am familiar with the subject matter, as I can prompt for refinement and improvement. It is kinda following the Karpathy videos so far (a couple lessons in) but adding more math/derivations, which was what I asked for. It has trouble staying on topic, presenting information in a coherent/meaningful order, and providing all the context necessary to move through steps in its "course notes".
Like I said, I'm essentially continually prompting to refine the material. LLMs certainly continue to append, and never cut back. It just keeps spitting out additional content at me. So that's a bit annoying too. But I can basically get figure out what's going on with a few extra promps.
If youre curious what i've got so far... just be warned it is quite literally AI slop plus me continually prompting for clarification/cleanup etc. : https://github.com/cmoscardi/ai-for-ai
novaleaf 16 hours ago [-]
opus 5 doesn't follow instructions that great. probably needed that extra "creativity" to benchmax. if you are stuck on anthropic, try Opus 4.8 or Fable5 a try with the same prompts. very different results.
It generates tutorials for you, and serves a webpage that lets you complete them. It does a remarkable job.
It still has a bit of the LLM prose problem, but it does help you fine tune the ‘voice’ it uses.
SambhavGupta 15 hours ago [-]
To get rid of the llm prose issue you can take a representative sample of its prose (say a question and its answer), rewrite the answer in the way you'd prefer, and add that to the system prompt; I did this to get my LLMs to compress down what they say, and now everything they say is very dense and to the point
agumonkey 16 hours ago [-]
anybody adds some prologue about what style of answers are prefered ? i know i often try to change the linguistic patterns because i too (unsurprisingly) am tired of llm prose.
esalman 14 hours ago [-]
I've heard people promoting LLM to use ASD-STE100 Simplified Technical English with some improvement.
treetalker 15 hours ago [-]
So you want it to teach you and take notes for you?
sscaryterry 16 hours ago [-]
> want to read anything generated by something like Opus 5 at this point
I've stopped using CC because of it. I find it insufferable.
puttycat 15 hours ago [-]
Opus 5's language is just terrible
dbtc 14 hours ago [-]
"hey LLM, I'm trying to learn about ___. I already know ___. My favorite authors are ___. Suggest some reading material."
biophysboy 16 hours ago [-]
It annoys me that the default AI mode is so tedious and longwinded. I read a lot of nonfiction - the house style of AI is basically marketing copy.
unclebucknasty 14 hours ago [-]
>LLM prose is annoyingly dense, and the useful information gets lost in a bunch of noise.
This problem doesn't get talked about enough and is second only to the hallucination problem IMO.
AI produces so much noise to wade through in order to find signal, and the more expertise you have in a field the more that costs. That noise directly subtracts signifcantly from productivity gains.
And, I think the problem is directly related to the hallucination problem. It feels very much like an effort to kitchen sink the response in order to provide some value among possible hallucinations.
It also seems to be a byproduct of Gen AI operation. It just fundamentally doesn't understand what it's outputting, so doesn't know how to narrow down to the most salient bits.
epolanski 17 hours ago [-]
> LLM prose is annoyingly dense
It's just long. It just doesn't shut up. It's overly verbose. And you can't tell it to be concise or you degrade its quality.
If I ask what an integral is, the correct answer is that it is the continuos analog of a sum, generally used to calculate areas and volumes.
It should really be a single sentence, and then let me ask more about the terms I don't understand, and here's the beauty, in the previous one there can be only 5 terms I cannot know.
An LLM will vomit an entire page or more of explanation which isn't bad per se, but is an answer to something different: "give me a short introductory explanation to integrals". And that's not what I asked.
onemoresoop 15 hours ago [-]
I call that vomit shotgun answers, text from which you have to filter out all the extra info the LLM wasn’t asked for. Luckily you can control that behavior and make it behave closer to what you want. Just ask the LLM how to ask for it.
oblio 16 hours ago [-]
The new Google Translate. They've made it slightly better at translating paragraphs of text but in many cases it's lost the basic function for translation: dictionary.
Try it out, fairly sure that if you out in 100 random words for 30 of them it will just refuse to translate them (it will copy paste the original word into the target language) or it will do silly things like use the target 4th dictionary definition instead of the primary one).
yieldcrv 15 hours ago [-]
Its aligned in getting you interested and frustrated about a topic enough to go to a primary source you would have never looked at
lanfeust6 17 hours ago [-]
LLMs help me refine my search. If I want to dive deeper into any topic, I can yield a strong list of primary sources relatively quickly.
jijji 13 hours ago [-]
something to try that works really well is to put "explain this as if you're talking to a 5th grader" at the end of your request.... it just breaks down the text into more manageable sentences that can be understood by general audiences
colordrops 14 hours ago [-]
You can instruct it to be terse and even take on a specific voice if the standard prose bothers you.
misiti3780 17 hours ago [-]
I think AI is surprisingly good at this. I use voice mode while working out to learn complex topics, follow up with reading, and then go back to ask the LLM more questions. They excel at simplifying complex ideas and have endless patience. One hack I found is telling the LLM to test my knowledge by asking me questions—that gives me a clear idea of what to read next. Overall, they’re a great tool to use alongside traditional learning methods like reading books and working through practice problems.
FomoFox 12 hours ago [-]
[dead]
mancerayder 16 hours ago [-]
[dead]
imadierich 15 hours ago [-]
[dead]
nullsanity 16 hours ago [-]
This is on top of the issue of LLMs being a moron. like, I'm sure dumb people can learn stuff from them, but I'm sticking to books written by people who know what they're talking ahout and not vibed slop.
runarberg 15 hours ago [-]
AI hater here. Too me this story is all to predictable. This is exactly how I thought using LLMs to learn would go. Or in simpler terms: well duh.
I'm actually going to make a prediction here as well. I think you will soon realize that using LLMs to clarify certain questions or ideas you have will turn out to have frustrations as well. And that you will soon direct those questions to either peers you know in real life or internet forums which are very likely to have a non-AI policy.
andy99 14 hours ago [-]
> I'm actually going to make a prediction here as well. I think you will soon realize that using LLMs to clarify certain questions or ideas you have will turn out to have frustrations as well.
Much more likely that people will believe themselves to be an expert in a subject after having had a conversation with Claude about it.
knollimar 13 hours ago [-]
Wild because people wouldn't do that with a friend. Something about an LLM giving people info has this insidious stoleb valor to the LLMs work. Like that wasn't your thought, you just google searched it.
varispeed 17 hours ago [-]
> First, I get exhausted reading LLM prose.
Often "be concise, to the point." is enough, but you can also paste it some stuff you like as an example text and ask to do style transfer.
j45 16 hours ago [-]
You can change the prose that it outputs to anything you like. ask the llm how.
j45 14 hours ago [-]
Specifically, you can set the tone and style to match what you like or used to after asking it to interview you, create an output, and then use that output as the project description or document to refer to
bluejellybean 13 hours ago [-]
This feels like the weirdest complaint to me. Just tell it to shorten its response.
Literally saying "one sentence response" solves most of this problem.
wxw 18 hours ago [-]
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
I'm not sure I follow how this is actually guaranteed? The fact-checking process mentioned just seems to involve asking AI to review its own work.
dozerly 17 hours ago [-]
All these LLM-as-review hype pieces don’t acknowledge that it’s turtles all the way down
enraged_camel 16 hours ago [-]
What do you mean by this?
Gander5739 16 hours ago [-]
If the output can't be trusted, and you use another llm whose output can't be trusted to check the untrusted output of the first llm, then you're back where you started.
duncangh 15 hours ago [-]
Yeah this seems to me similar to how the mortgage backed security risk concentration occurred leading up to the global financial crisis. Whereby the risk from exposure to low grade / risky single mortgages was eliminated via diversification but the diversification was simply packaging all of the risky MBS’s together and in no way diversified or de-risked the entire portfolio
Terr_ 14 hours ago [-]
I'm hoping that the Big Horrible Realization comes sooner rather than later, when we have less collective damage and pain riding on it. (Plus I'd feel personally vindicated.)
cromka 14 hours ago [-]
I don't see it. To me it's like having e.g. 3 drunk PhDs arguing between each other to settle on truthful answers to questions.
DrewADesign 13 hours ago [-]
But the problem with LLMs is that they get the facts wrong. PhDs are PhDs because they’d look it up in an authoritative source, or actually find out through research and experimentation. The whole point is that facts aren’t a matter of opinion. The only people that argue over documented, findable facts are idiots that nobody should listen to.
northern-lights 6 hours ago [-]
Unfortunately, I don;t think that can ever be fixed. For an LLM to know that it is not hallucinating about something, it must know that that statement(s) is/are true. Which it cannot infer due to Godel's Incompleteness theorems.
ex-aws-dude 10 hours ago [-]
No you don’t get it, I asked it specifically to make sure it’s accurate
daishi55 15 hours ago [-]
Not really. Take hallucinations for example. If they are 1 in 100 (actually they are much rarer, but for the sake of argument), then the chances that 2 LLMs or even just 2 runs of the same LLM have the same hallucination is, well, a lot less than 1 in 100.
Terr_ 15 hours ago [-]
That rests on a false-assumption that the errors are statistically independent events, and have nothing to do with the shared nature of the judges.
Systemerror7A69 7 hours ago [-]
It's also relying on the assumption that the checking LLM only ever corrects wrong statements and never incorrectly "corrects" an already correct statement, which might not always be the case as well.
daishi55 14 hours ago [-]
Are there any reproducible hallucinations on any of the currently available OAI/Anthropic models? I’m not aware of any.
And even if they are related - if Opus 4.8 always has a 1:100 chance of a specific hallucination - then running the same model twice does indeed dramatically reduce the odds of an error in the final output.
Terr_ 14 hours ago [-]
If simply running things thrice-over was enough to stop "hallucinations" (and not incur other problems) we wouldn't be here talking about it today, it'd have been "solved" months or years ago.
fluoridation 7 hours ago [-]
Even if your conditions are granted, different LLMs can easily consistently hold divergent "worldviews". LLM A might consistently state that fact P is true (i.e. it will never hallucinate a string that states or implies ¬P), while LLM B consistently states that fact ¬P is true. You don't have a way out of this with these two, and adding a third LLM is an additional confounding factor, not a solution. You can't settle questions of fact by survey, you need to test or use reliable sources.
djhn 9 hours ago [-]
Constant hallucinations. OpenAI:s latest on max settings. If you are to naively feed say, a short corpus of text to turn it into a parallel corpus in a few different languages, the original text gets subtly mangled and no longer matches the original. Say you have several hundred annotated sentences. Without hand-coding some regex to make sure that each sentence in the source column occurs in the original corpus you’re bound to get hallucinated sentences with an error rate that exceeds 1:100. Whatever you use as the output, JSON or XML, you will end up with columns that just repeat the original instead of translating it, especially for languages that are very close to each other or represent the same language.
Yes, LLMs can be SOTA for NLP, but you’re going to have to use them to write software or workflows that are more deterministic.
15 hours ago [-]
furyofantares 16 hours ago [-]
"Turtles all the way down" is a phrase of, I think, unknown origin (https://en.wikipedia.org/wiki/Turtles_all_the_way_down) about infinite regress or trying to patch up some bad theory by appealing to itself. Someone claims
that what holds the Earth in place is that it sits atop a giant turtle, and a skeptic asks what holds the turtle up, and the response is that it's turtles all the way down.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
dozerly 15 hours ago [-]
There are certainly uses where it’s good enough, but you can never be 100% certain of correctness in the way that people claim you can by stacking N layers of these models.
What triggered my response was the “just review the output with another LLM and it’s perfectly correct”
TooSmugToFail 7 hours ago [-]
I would have bet it’s a Terry Pratchett quote, and it kinda is:
“"The turtle moves," said Didactylos. "The turtle is a giant reptile that swims through space. It doesn't have to stand on anything. Swimming is what turtles do. The idea that it has to stand on another turtle, and that turtle has to stand on another turtle, is just silly. It's turtles all the way down, and that's a logical absurdity."
Small Gods, 1992
phatskat 16 hours ago [-]
I read it as “if your LLM is being checked by another LLM, well then you need another LLM to check the checker. And can you really trust _that_ LLM? Probably should have an LLM to check the third one, and…”
tclancy 15 hours ago [-]
It’s a reference to Bernard Shaw, who once said that if we ever created a truly artificial mind it would be inside a turtle’s shell. Sturgill Simpson covered the track on his seminal work, Xeno’s Paradox.
Dumblydorr 13 hours ago [-]
Might be a reference to the story at the beginning of A Brief History of Time, attributed to Bertrand Russell’s audience member.
Full story in the book
tclancy 12 hours ago [-]
This guy remembers what I thought I was saying. In my defense I stole the whole thing from Stephen King’s It which I read … forty years ago, that can’t be accurate. Let me sort out my instruments and get back to you.
eyepea2007 15 hours ago [-]
an LLM tells me there is no evidence that Bernard Shaw ever said any such thing :)
Agreed. Given how many significant errors LLMs make in my topic of expertise, despite my taking multiple error checking steps, the idea of catching 100% of hallucinations because you told the LLM to check itself is hilarious. It’s just a wild lack of insight: “I’m using the LLM to teach me something I don’t know about, I definitely have the knowledge base to spot any errors that might remain!”
DrewADesign 15 hours ago [-]
Yeah. People with technical and/or tech business bonafides claiming that AI granted them expertise are so often taken at face value when they really shouldn’t be. Who told them that they were proficient — a chatbot? Someone who knows even less about the topic, so any expertise seems impressive? I’ll bet it wasn’t someone that actually knew what they were talking about. Even some tech reporters are tripping over themselves to be amazed, but don’t bother checking if they should be.
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
latexr 4 hours ago [-]
[dead]
nedt 2 hours ago [-]
I'm just looking at the rocker engine piece. The belt is running backwards. That's the first thing I'd expect it to get right or flag if it doesn't. Human in the loop is also the quality control. If that fails the rest might have similar issues.
hn_throwaway_99 17 hours ago [-]
Completely agree. While I didn't set things up to have AI review its output in a loop, my experience trying to build a specific acoustic testing rig with Opus 5 also aligns with the other "it's turtles all the way down" comment.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
knollimar 12 hours ago [-]
A second agent reviewing it adversarially resolves some context rot. Whatever they trained these LLMs on will just infinitely double down so I break it with 1 layer of checking and then a judge who looks at facts, since the checker is adversarial.
I know it sounds silly but 1 layer ends uo being way worse than 2.
bryanrasmussen 17 hours ago [-]
I could certainly envision a scenario whereby review would increase reliability but not how it would every guarantee 100%, there is a pretty big logical gap there.
laurentiurad 17 hours ago [-]
In my experience it depends on how much in detail you want to go. Chip manufacturing is a really opaque industry, so in this particular case LLMs might not even have the training data. However, using it for a high-level introduction into something is usually pretty safe from hallucinations.
spike021 15 hours ago [-]
even if you say use RAG or something to a source you can trust, there's no guarantee the agent will still use exactly what the source has.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
SwtCyber 9 hours ago [-]
The "100% accurate and free of hallucinations" claim should probably be replaced with something much weaker
15 hours ago [-]
zahrevsky 13 hours ago [-]
I don't do animations, but I have an answer. You research a topic well enough to be able to understand if the result is OK or not. Usually it means figuring out some sort of testing.
I'm researching causal inference right now, and my main goal was to make sure I understand how to test estimation on synthetic data.
Basically, it's the same way it works with people. If you delegate a task that you don't understand, and you can't have a credibility proof (i.e. doctors, lawyers), then you research a topic well enough to be able to (1) define the task and (2) verify the end result.
63stack 14 hours ago [-]
You can add "make no mistakes" to the end of the prompt and achieve the same result while burning less tokens.
lowsong 14 hours ago [-]
Yep, that's impossible. The hard truth is that most people this lost to LLM psychosis cannot understand that fact. It's better to treat it like someone in a cult, arguing the facts isn't going to help if they refuse to accept them.
a3w 17 hours ago [-]
I thought after reading the title that the text was about learning something, yet the actual text seems to be about having a system do something for me.
gnarlouse 13 hours ago [-]
I've had success using the socratic method. I give Claude some topic (say, how the intricacies of the bond market works, the content from which are screenshots of pages from a textbook), and then I go on a walk chatting with it in voice mode. Claude is the expert, I am the student. Claude asks me questions, leading me to an answer logically. I come back with questions, and we back and forth. LLMs arent like they were in '23-'25. I'm almost always skeptical its going to lie, and almost always wrong.
In particular:
- I limit it/encourage it to give me single sentence questions
- I sometimes will ask it to tell me a motivating, human-grounded story, when we're starting a new concept: claude responds "Maya is a bond portfolio manager, and her boss has asked her to quickly price in what happened if yields go down. She knows her bond's average duration, a measure in time, but she doesn't have a percentage, which is what her manager wants. How can she give him a percentage number with just a duration figure and the proposed new yield?"
- I'll often ask claude to let me work through it, to derive the thing myself, often resulting in a string of thoughts with "yes/no" trailers, to get the LLM to reply yes or no only, and avoid derailing my train of thought. If yes, my train of thought keeps going. If no, I've got something wrong.
- I'll sometimes stop and have it craft an artifact. I typically say "build me a Brilliant.org-style interactive demo of the topic", especially when we get into the realm of looking at the actual maths of a thing (for which prose and dialog is not optimal by itself AFAICT)
- I'll do this while I'm traveling, while I'm walking, while I'm doing chores.
It's so much fun.
pumanoir 12 hours ago [-]
can you share ane example of one of your prompts? I have tried to convert some text into the socratic method and haven't succeeded yet.
gnarlouse 12 hours ago [-]
It's a stupid word soup on second read, but hey, it works, and that's all that matters with "stochasty the parrot."
"""
In this project, I require a socratically delivered line of conversation. Here's the typical structure to the conversation. I ask some question. You need to factor and reason about how to conduct and deliver a conversation. Best practices would be to limit terminology, or assess with the user whether they have a firm grasp on terminology before you use it. You must be very strict about this, it's unacceptable to just introduce a new concept, actor, phrase or other complication into the conversation without first labeling who what or why it exists for the conversation.
Conversation structure needs to be front-loaded with a brief interview for the user, "you understand X?", "whats your understanding of Y?".
Conversation structure then needs to proceed with single-sentence questions from the agent. User replies with an answer. Sometimes the agent needs to correct the user, but only ever do so with yet another question.
"""
^^ these are the instructions I have installed at the root of a "project".
Keep in mind, this is claude opus 5 low effort we're talking about, in the "projects" area of the mobile app. Here's the process I use to set up its knowledge:
1. I take screenshots of the textbook on my iPhone, and upload a chapter at a time.
2. I have it summarize the chapter into markdown by analyzing screenshots. You could probably achieve this simpler, if you just had the textbook in PDF.
3. I walk with my boy Clau-crates.
I've done this for a couple of weeks and haven't seen it revert back into its typical context-dumping behavior.
On second read, there's probably some clean up I could do. Thanks for making me pull it out and look at it. Things that could probably be improved:
- tell it to cross-check resources online to further ground itself
- use simple, short sentences (long sentences make the brain blur a bit)
- not be sycophantic (it seems like project mode has discarded with my root-level anti-sycophancy prompt)
URMAJESTY68 53 minutes ago [-]
> What you get is a beautiful animation that is 100% accurate and free of hallucinations
I can appreciate using all of the tools at your disposal to learn a new topic, and in no way want to discourage learning. I've used LLMs myself to question my own understandings and it can be helpful.
However, "... 100% accurate and free of hallucinations." isn't a statement someone who just learned the topic is capable of honestly stating.
f311a 18 hours ago [-]
What’s everyone’s opinion on learning new tech things in this day and age?
My opinion swings between positive and depressing vision of the future.
I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
For example, I’m pretty good at optimizing low level stuff, but right now you can just ask LLMs to do so and they are pretty good at it. They will profile the code and suggest reasonable options like 90% of the time.
eterm 18 hours ago [-]
They're amazing at it, provided you keep asking the right questions.
Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.
You might think what you're doing is trivial, it may be sessions that flow roughly, "Instrument this, okay this part is slow, profile this part, OK read the profile output and suggest a better approach".
But your experience will be steering it in the right direction, and you're probably unaware of just how much your experience is doing that guiding, as the LLM shoots off at 100mph, you feel like it's taking you with it, but you will be guiding it a lot more than you realise, and that's where learning and experience comes in, even if you're no longer operating at the lowest depth, your knowledge of that layer will be helping.
If nothing else, the experience to know when something is actually slow is a skill in itself. If a function takes 200ms, sometimes that's as quick as it can realistically go, and sometimes that's literally a million times slower than it could be, and there's actual skill and experience wrapped up in knowing what "slow" looks like.
jack_pp 7 hours ago [-]
> Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.
So there's this 11 year old issue in a forgotten ffmpeg plugin and I fixed it with deepseek by putting it into a self-testing loop. Probably would have taken me a few weeks to even understand the initial code to begin with. I haven't done C work in a long ass time and have zero knowledge of even what sub pixel sampling means.
With DS4 took me a few days and a couple of dollars. And by few days I mean I checked on it a few minutes every half hour or so a few times.
I don't understand the code it wrote but it's been in production for a while now and no issues so it's good. Ended up speeding up our video processing pipeline by 20-30%.
9dev 6 hours ago [-]
So while I do agree that it is cool you were able to do this, I‘ll say:
Sending in the patches but refusing to take responsibility for them is a surefire way to contribute to maintainer burnout. Please don’t do this. Either commit to fixing something and driving the PR to merge, or abstain from it entirely.
The bottleneck isn’t the speed of coding, and what you’re doing here is actively worsening the situation.
jack_pp 32 minutes ago [-]
idk if it's really worsening, someone who has the same issue can apply the patch. I understand why they rejected it but also I'm not willing to learn everything that DS4 did there since I don't plan on working on ffmpeg filters and the knowledge is pretty much useless to me outside this patch. Too much time commitment for.. fixing a filter no one really cared about for 11 years.
But you can see how the parent's comment doesn't really hold, I was able to achieve this while not knowing anything other than what I need fixed and making the LLM test itself towards that goal.
therepanic 13 hours ago [-]
So what should newcomers do who haven't yet gained enough expertise to ask LLMs the right questions?
lfx 13 hours ago [-]
Newcomers have to do exactly the same what they did years and years ago. Practice the craft and gain experience by doing things.
And the craft is loose term, it can mean anything you like to get better at.
wonnage 17 hours ago [-]
“asking the right questions” is built on years of experience doing the things now being offloaded to AI
“asking the right questions” is also a moving target with each model release
People simply underestimate the value of doing the work and think that the end result is all that matters
RunSet 15 hours ago [-]
> “asking the right questions” is built on years of experience doing the things now being offloaded to AI
Except the steering itself is also disappearing, the same prompts from just 6 months ago now need much less steering, AIs are learning to even ask back in certain cases to persuade people with no experience towards the most likely correct choice
tunesmith 17 hours ago [-]
Generally speaking, the pattern is that people are overestimating how much "work replacement" will happen, and underestimating how much "work shifting" will happen.
What is fascinating is how you can witness it at so many levels of organization. One example: Employer executive get enamored with moving from labor to capital. They believe that by using LLMs, they can replace a lot of workers. At my place of employment, we have people that are surprised they can't file a Jira ticket describing a product ask, and have it kick off an implementation. You can build the skill to attempt that, but invariably you'll get back questions like "what do you mean by <x>" and "what do you want to do in this case, a, b, or c?"; questions that a product person or an exec are not well suited to answer.
In the past, programmers did that kind of interpretation and judgment call. So then you're in a quandary; who should do that work? Work that previously, you never imagined was an inherent part of what the replaceable code monkeys do at your beck and call?
And then, how do you hire for that? How do you find the training for the people that are experienced enough with... something... to know what a cohesive error response is, or what kind of telemetry strategy is best for that particular product and organization, what collection of product asks are incredibly complicated for what they're asking and can deliver 95% of the benefits at 5% of the work if we just do this instead, and whether you want to aim more towards thick or thin clients?
Who are those people? Wait, those are programmers? Wait, there's this whole collection of inherently human skills that we devalued, by not appreciating they were always quietly doing that for us in the past?
That's just one example. There's a repeating pattern of discovering where the work truly is, work that was embedded in manual patterns we might not have to involve ourselves with anymore, but is yet still essential. So the nature of our jobs changes massively, but the overall level of employment does not.
At least, not in the medium to long term. There is a lot of painful churn we have to suffer through first.
Calazon 15 hours ago [-]
I agree with you in the broad strokes, but replacement by LLMs isn't the only way the level of employment can be reduced. If LLMs can make it so that two programmers can do the work that used to require five, that can result in a very large reduction in employment, even while having (some) programmers is still essential.
killingtime74 10 hours ago [-]
Only if there is no further additional demand created by the now lower cost of doing the work. Take lighting as an example. As it has progressed from burning expensive candles to now leds, our demand for lighting has continued to increase.
tunesmith 15 hours ago [-]
On the bet that layoffs will result in programmers ending up at 40% of the staffing level they are now, I will take the over.
Calazon 14 hours ago [-]
What sources do you think are good for this data for the recent past? My brief initial search is turning up a lot of contradictory data, probably because the sources are defining "programmers" differently from each other.
vilfred_sikker 53 minutes ago [-]
I'm also trying out a lot of different approaches, and generally I like the sparring I can get from LLMs but with the acceptance that they make mistakes and some things still need some fact checking. But the models are improving fast, so I'm positive about this approach just becoming better going forward
I'm working on a side-project called tech-professor.com which is a platform for learning. The content is built directly from the source code in your pull request and repositories you follow. So you can quiz yourself and your team based on the code you ship.
It's in beta and a lot of changes are still on the way, but if you want to check it out and give feedback, feel free to sign up for free.
cautiouscat 18 hours ago [-]
I can only speak for myself, so I hope this resonates with you.
I wasn’t even really concerned with optimizing low level code before LLMs and that wasn’t why I was hired either.
However following that low level thread: We can look at the reasonable options and immediately know if they’re reasonable or nonsense. Why? We know the code. Now zoom a level out, where I think our expertise really lies.
Building a complex system isn’t easy. There are customers with requirements, there are budgets, SLAs etc. Sometimes one customer needs X and one needs Y. Our expertise is taking all of this in, and producing something that balances all the different variables. It’s knowing that we’ll expect X events a second so we’ll need Y to ensure we can tolerate failure.
Is it possible LLMs will be able to do all of that too? Maybe. But then why would our customers need the enterprises they pay for?
SkyeCA 18 hours ago [-]
> I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
I have stuff to do now, the value of the knowledge in a year or two isn't important if it solves the issues I have today.
AnotherGoodName 17 hours ago [-]
I sometimes get vague ideas for solving maths/science problems. They never pan out but i can talk in detail on group theory and advanced maths and science topics due to investigating such vague ideas over the years. These days the LLM shoots the ideas down instantly and honestly correctly, i know enough to know "yeah that's right, oh well" and move on. Which actually takes away a huge avenue of learning. I'm pretty torn on the outcome of this honestly.
I'm not 'wasting time' but I'm also not really learning.
le-mark 12 hours ago [-]
I felt this. I had a feeling for a long time that spider solitaire was somehow related to knot theory (legal moves being Reidemeister move equivalents and untangling). Llm shot this idea down very quickly! It was freeing actually.
fluoridation 7 hours ago [-]
Both of you are being overly dismissive of yourselves. Without either a proof or a program that proves the statement in question, I wouldn't trust the LLM. I wouldn't even trust if it produced citations, because I've already come across LLMs citing sources to back up statements without being able to find that information in the source (not saying it wasn't there, just that I was not able to find it).
simonw 17 hours ago [-]
There's a big gap between being able to ask questions about something and understanding something.
The more things you understand, the higher the chance you'll spot a situation to use them in the future.
I think the best innovations come from times when someone is uniquely able to combine two of their previous experiences together. The more experiences you have in your back pocket the more combinations you have access to and the more likely you'll have a unique combination when the right problem comes along.
yoz-y 2 hours ago [-]
Purely personal but as I get older I am tired of learning new things and would just rather learn old stuff.
Career-wise, this is terrible. But for me technology was polluted by the ever-growing greed.
For example I’d much rather actually learn assembly, than learn the nitty-gritty details of how LLMs work.
shannadige 18 hours ago [-]
There’s still an immense value in training the brain to learn and be able to approach new problems with the sort of procedural thinking that LLMs enable. We can explore topics that we are curious about and develop that sort of “muscle” to continue asking questions when we have them. I have no fear that when those bigger (and existential) problems arise we’ll be well equipped to keep asking questions and figuring out ways to solve them.
tarkin2 10 hours ago [-]
Not good at subjects you have no previously learnt knowledge.
They tell you have "hit the nail on the head" when you really haven't.
They tell you have had a "great insight" when you are really haven't.
They give you the illusion of learning and progress but essentially give you faulty preconceptions will trip you up further down the road.
You can ask the LLM to be more critical and less sycophantic but that only gets you so far:
They want you to continue using, being dependent on and feeding data into the LLM--your independence isn't a priority.
brudgers 18 hours ago [-]
One possible value in learning new things is developing a habit of learning.
In particular it might be valuable to be in the habit of learning things that one is bad at doing.
Or not.
maplethorpe 12 hours ago [-]
I'm choosing to (mostly) switch off the news and continue learning things I find interesting anyway. Maybe the world will punish me for it at some point, but I guess I'll have to deal with that when it happens. The alternative is too depressing otherwise.
numpad0 17 hours ago [-]
So you think building a warp drive is pointless because 99% of work, per your judgement, will be done by AI? Is your contribution meaningless and artifact useless? I wouldn't think so.
kccqzy 16 hours ago [-]
It depends strongly on the prompt. Like recently I was also doing some performance optimizations, and if I did not mention profiling none of the LLMs even profiled the code; they merely read the code and assumed, based on their own analysis of big-O time complexity. It is after I explicitly asked for profiling that the LLM started actually profiling.
SketchySeaBeast 18 hours ago [-]
> I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
This is silly. This would be like arguing that encyclopedias made knowing things pointless. I learn new stuff for me.
Professionally, it's important to know enough to know if you're going in the correct direction. Practically, tokens are going to continue to cost money and knowledge can save you tokens.
8bitsout 18 hours ago [-]
I mean, part of the reason the LLM can do that is because you know enough to direct the LLM to do so and verify the results to some degree, right? It's good to learn new things because:
1. It satisfies you curiosity (and curiosity is always valuable)
2. You can better utilize the LLM to expedite something you now have knowledge about
3. You still improve as an engineer/programmer/prompter/whatever
I still think it's very important not to outsource everything to AI because there is a lot of value in learning and doing things yourself which is an important part of life.
catlifeonmars 18 hours ago [-]
I don’t agree with the implication that there has to be a practical reason to learn new things. I enjoy learning new technologies because it’s fun.
laurentiurad 17 hours ago [-]
The way learning new stuff rewires your brain cannot really be predicted, but the effects are positive.
jazzyjackson 18 hours ago [-]
I decided to learn watchmaking instead
ofjcihen 16 hours ago [-]
I have staff ranging from 10 years of IR experience to right out of college.
I can tell you that there is an enormous gap in ability between them despite them both using LLMs for daily IR work.
The reasons aren’t complicated. The senior responders have tacit knowledge of how breaches evolve and what to look for which gives them a much better framework for where to employ the LLM.
The juniors will normally start from “here are some logs, look for weird” which is fine but leads to tunnel vision and a lack of confidence in their reporting.
I don’t mandate that anyone do work with or without an LLM. I hire seniors based on experience and juniors based on interest. But my experience has so far been that our best up and comers focusing more on learning the technologies instead of leaving those details to the LLM are developing their intuition and understanding faster and in a more robust manner.
einpoklum 2 hours ago [-]
If people don't learn, and write, and converse, about new tech, the LLMs will not have raw material to learn their slop from. And I am quite worried that this is the direction we're going in, with the lake of insight being sipped dry and a future of shallow cliches as LLM responses.
phailhaus 18 minutes ago [-]
The fact the author thinks every topic can be fit onto a rollercoaster tycoon-style analogy leads me to think they do not actually understand these topics very well.
royal__ 18 hours ago [-]
The biggest thing I've learned from doing stuff like this is that there are no shortcuts. At some point or another, to truly learn something deeply, you've got to dig in to the boring details and do things the hard way. LLMs can help with this...but I find it's usually tempting to try and just offload the boring stuff to them, which doesn't work.
mfalcon 1 hours ago [-]
The LLMs are a way to get quick feedback and getting it promotes quicker learning (if the feedback is good). Learning is difficult, sometimes I get blocked on something and before llms that could prevent me from learning some material, nowadays I have another tool to help me unblock and learn faster.
I think that could be called a shortcut, not in learning per se, but in the process of getting to learn.
hahahacorn 18 hours ago [-]
Having an initial higher level understanding across the domain is extremely useful to contextualize the deeper stuff. I think boring details is a very leaky characterization, but I'll continue with it.
In my experience it's infinitely easier and faster to learn deep, "boring" things when you understand how they relate to your shallow and wide understanding of all of the related components.
The LLM is merely a tool. And you can use it for domain discovery that enables efficient deep learning at an unprecedented rate or you can develop a cursory understanding of a topic and think yourself an expert.
yoyohello13 17 hours ago [-]
I’ve found that if I’m not struggling I’m not learning. If stuff is coming fast and easy that’s a sign that what I’m doing is not stretching existing skills enough.
It’s true of most things. Running, dieting, weightlifting being uncomfortable is a sign of progress.
djaro 8 hours ago [-]
This is also why apps like Duolingo don't work for most users. They want to turn language learning into a fun, effortless game but effortless is fundamentally incompatible with learning.
Learning is uncomfortable. Reading a difficult (for you) text in a language you don't understand is exhausting and confusing. But that is where improvement happens.
Its even worse since Duolingo added a life system (not sure if they still use it), where you were only allowed to make 3 mistakes before having to recharge your energy. If you get everything right, you are not learning, you SHOULD be making mistakes constantly. That shows you're actually being challenged.
hlovdal 4 hours ago [-]
> If you get everything right, you are not learning.
One of the teachers on the college I went to had a note on his door with "If you understand everything you're doing - you're not learning anything.
AdieuToLogic 13 hours ago [-]
> The biggest thing I've learned from doing stuff like this is that there are no shortcuts. At some point or another, to truly learn something deeply, you've got to dig in to the boring details and do things the hard way.
This is the only path to mastery, or understanding if one prefers. There are no shortcuts to a person achieving deep understanding (a.k.a. "Aha!" moments).
Can a tool such as GenAI be beneficial to someone who already has done the work to understand? Absolutely. But it cannot infuse mastery into a person simply by its use.
Only the time and effort a person devotes can do that.
ainch 18 hours ago [-]
Quite - you still have to do the work yourself. I think LLMs are best placed to act as an eager tutor that doesn't mind discussing a topic ad nauseam until you're certain you understand it.
mhh__ 17 hours ago [-]
yes i've found that there are a few topics that i've really been able probably 10x my understanding of using LLMs, in particular in getting me over hoops that are hard to navigate when solo, BUT I have to be really careful for it to not just show me the answer all the time.
muh_gradle 13 hours ago [-]
> In plan mode (using CC, or OpenCode) I ask a model to build the foundational knowledge for X topic.
Makes sense.
> I ask it to review the accuracy of the knowledge base it built in the previous step.
Ooookay that sounds good.
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation.
wat.
lfx 13 hours ago [-]
Author likes this kind of animation. I have something similar like this, but without the animation step.
The main idea is you can do any style you want or like to learn.
ch4s3 13 hours ago [-]
It's such a surprising and delightful turn, I love it!
rickcarlino 18 hours ago [-]
I’ve been using LLMs to create readable rewrites of RFCs and specs that interest me. It is not precise enough for implementation use, but it has increased my understanding of the underlying RFC.
Another useful approach has been asking Codex to implement complex things, like a Kademlia DHT or BitTorrent client in a literate style with the explicit purpose to increase understanding by reviewing the source code.
I only use LLMs for explaining things if the subject is one I’m deeply familiar with and can independently and easily verify the facts. Doing so with new knowledge areas is very risky for obvious reasons.
For example, “explain how the code in this file works,” I am familiar with the overall codebase, I know the purpose of the file, and I can read it or write tests to verify if I suspect what it’s telling me isn’t correct. Or, if it’s really important, I can overcome my introvertedness and ask the team member who wrote it…but that’s a last resort nowadays, which I am very thankful for. In 99% of cases since at least Claude 4.2 days, Claude and Codex have been very accurate. Gemini on the other hand messes up more frequently and sometimes does weird things like try to delete files it’s not familiar with, at least the 3.6 flash model I’ve been using lately does this. But, code explanations are still good for the most part.
controlbird 12 hours ago [-]
100% agreed.
vilfred_sikker 1 hours ago [-]
Really cool idea the sim game approach! I'm trying to find a good approach to learning in the age of agents, though more focused on the actual code I read and write in my day to day. I'm building tech-professor.com which is in beta, where my goal is to setup continuous learning for product teams and developers.
It's a hard problem to solve, but I believe that llm-based learning will become better as models improve!
ventana 18 hours ago [-]
The title is not representing what the post is about. “Use LLM to learn complex topics” here actually means that the author asks an agent to describe the problem area, and then implement a simple web-based simulation game, and by playing that game, the author actually learns about the topic and its constraints. They use chip making as an example.
That's actually a fun way to learn processes!
lacedeconstruct 18 hours ago [-]
Its fun but is it really effective ? I mean I checked the LLM one and I came out more confused about a topic I already know about, I find the best way to to learn using LLMs is to just generate an example try to somewhat get a mental model of how it works and then ground my understanding with traditional documentation and resources, its an iteration of a technique I used to do in college where I would read the textbook questions first to understand what is important and then read the chapter
ventana 18 hours ago [-]
I assume different ways of learning work for different people. For me personally, it's taking a piece of paper and drawing the diagram of how things work together; of if it's some math, then, again, using the pen and paper to follow the text. I can very much accept that for some people playing the simulation is a good way to touch the new problem space. I can easily imagine that for some topics, let's say, traffic signal automation, a careful simulation game will probably give more information than reading papers or manuals.
lacedeconstruct 18 hours ago [-]
> a careful simulation game
Totally agree, unfortunately careful simulation games are very rare
sixtyj 18 hours ago [-]
Game-based learning, described by Comenius, works if someone else prepares “a game” for you. E.g. like a dungeon master. :)
Otherwise you probably get more confused as you have mentioned.
On the other side, Peter Diamandis describes a situation where a bunch of kids were given a internet-connected computer and they had no teacher. Instead of it there was a “grandma” that checked kids from time to time.
After that there was a knowledge test that revealed “no teacher” approach was more efficient.
But it was a group, not an individual activity…
laurentiurad 18 hours ago [-]
Thanks! The original title was "How I use LLMs to learn...", but somehow HN removed the "How" part. I even removed the initial post thinking it was a typo on my end and tried to post again, but I stumbled upon the same behavior.
Anon84 18 hours ago [-]
HN likes to remove “How” and “Why” from titles. You need to edit them back right after you post it
EagleEdge 2 hours ago [-]
For me the bigger value comes from explaining complex stuff. I remember Andrej Karpathy once said (He was quoting someone else actually) "You can outsource your thinking but you cannot outsource your understanding".
I can keep asking LLMs to explain a complex topic until I get it. Ask to explain it 10 different ways, explain it using physical analogies, explain it using visualization. If I don't get it, just say that out loud so that they can keep explaining it to me in different ways. We can keep going that until I really get it. That is the value I get the most using LLMs to learn things, especially complex topics.
einpoklum 2 hours ago [-]
> I can keep asking LLMs to explain a complex topic until I get it.
Or you'll be confused by the slop it generates, which is very conversational but may be just false. Or - you could "get" something wrong and believe it to be the truth because the AI said so.
nightwalkerkg 16 hours ago [-]
My main issue with using AI as a learning platform is that unlike documentation, books, Youtube videos, there is not really a process of having someone "review the learning material". For example, I can always read the review of some book or ciriculum, the comments under a video, or if it's some for of open source documentation you can check the PRs and verify to some extent it's claim. With AI I can't really say what it has halucinated, because I am learning a new thing, I don't have that benefit of previously reviewed material.
ethin 15 hours ago [-]
This is (exactly) why I very strongly tell people not to teach themselves with an LLM. Particularly from the ground up. If you do not understand the domain, you cannot learn from the model because you won't know what questions to ask and it certainly isn't going to answer all of them for you.
bonoboTP 43 minutes ago [-]
The LLM doesn't have to be the sole single only exclusive source you learn from. It's an addition to a mix. Nothing is a single definitive answer. Same way as many other things, like Anki flashcards or Duolingo or some YouTube channel, or some tutorial website or indeed a coursebook or an audio course or whatnot. Use multiple sources and use the strengths of each.
j45 16 hours ago [-]
You can ask the LLM to connect the dots to the sources before displaying it to you so you can see the proof in line.
You can ask the LLM how to do this. Start with a topic you know well to get the mechanism working and trust it well.
nightwalkerkg 15 hours ago [-]
Of course, and that is what I do, but it becomes and additional mental and time consuming effort. As an dumb example, if I am learning a new programming language and trying to grasp some concept, I'll check the docs, find the section and read it. I trust that the source in the docs has been already vetted by other devs and the authors. But if I ask the AI the same thing, I then need to verify it's claims usually by asking it to check if the info is true, and then possibly opening the source link (lucky for me Claude provides the links in the desktop app as footnotes). It's not a question about AI, it's about the trust I have of this tool, it builds over time, but as soon as the AI makes an assumption or a hallucination we are back down to square one.
I assume this will become less of an issue in the future as there is more trust between the AI tools and me.
steve1977 2 hours ago [-]
"Learn complex topics". Yeah, not really. Get a cursory, superficial overview of complex topics? Sure. But that's about it.
markusde 10 hours ago [-]
"Complex topics" in this case means reading 22 AI-generated paragraphs that supposedly cover the entire chip manufacturing process. If the author seriously thinks this level of detail is complex then they have psychosis.
num42 6 hours ago [-]
There is something called tacit knowledge that cannot be easily learned through LLMs. LLMs are currently very good for research and theoretical knowledge. For example, learning how to make mechanical watches involves a lot of hands on, tacit knowledge. That skill would also be transferable to building other precision instruments. I don’t think we can currently learn these kinds of skills easily with the help of LLMs.
Even if you know everything about bicycles their mechanics, components, and how they work you still have to learn how to actually build, repair, or ride one through practice. Knowing, understanding, learning, and practicing are completely different things.
There are also niche areas of expertise that can take years to develop,not just to the point where you know the terminology and jargon, but where you understand the nuances of the field, can recognize the "unknown unknowns", and eventually have the ability to push the boundaries of existing knowledge. Maybe that is what we should really call learning: not simply acquiring information, but developing enough understanding and practical experience to contribute something new to the field.
Ironically, on the same front page of HN, there is a post about Andrew Wiles and this. I don’t think I would be able to comprehend Fermat’s Last Theorem, the Poincaré Conjecture, or Gödel’s Incompleteness Theorems, even with the availability of LLMs.
dbreunig 13 hours ago [-]
> What you get is a beautiful animation that is 100% accurate and free of hallucinations
How does he know?
jmuguy 17 minutes ago [-]
I think these same people believe everything they see on TV, unless the thing on TV is something they already knew about.
dbreunig 13 hours ago [-]
Wrote about this awhile ago, and it hasn’t changed:
> Time and time again, when talking to people who rely on ChatGPT, Claude, Perplexity, and other general AI tools, I hear them say, “AI is incredible. It handles nearly everything I throw at them.”
> “What does it fumble with?” I’ll ask.
> “Well, it still gets things wrong when it comes to my line of work.”
It's the kind of overstated unjustified claim an LLM would make.
QwenGlazer9000 13 hours ago [-]
I thought of the "does he know" meme when I saw this.
ggm 13 hours ago [-]
Gell-Mann Amnesia. Recognises the failings in their own field of discourse, assumes all true about anything else, same source.
m0rc 7 hours ago [-]
For learning, I find LLM's helpful as a knoledgeable "tutor" or "friend" during the reading of a complex paper or book. Before, when I got stuck, I spend lot's of time going around different books/pages/papers until I was able to find a simple enough explanation that allowed me to "catch up". Now, I can ask the LLM and then continue from there.
Learning from a book is still the best... Although I have been told that learning a complete subject from a book is now an inproductive use of my time... Perhaps they are right, but I still do it.
qnleigh 6 hours ago [-]
Learning is an unproductive use of your time??? No one knows how LLMs will change things, but that sounds like terrible advice.
freefaler 8 hours ago [-]
Yes, there are many misconceptions on "learning", but LLM can be useful.
The approach that works for me is using Justin's skycak methods he mentions in his books:
Check the shorter "Advice on upskilling" or "The Math Academy way" for well researched approach.
So what works for me
- open a project in ChatGPT/Notebook LM
- dump all the relevant and highly cited materials (textbooks, papers)
- dump the advice on upskilling text or a short summary I've written for the LLM
- create "Learning Goals", that contain what I want to learn, and how to estimate is my level good enough
1) Ask it to create a learning path from the materials, following the approach. Give that to an adversarial LLM for cross check. (just for sanity check)
2) Ask it to create an "entry test" to check what I do know and what I don't
3) Iterate step by step on each module/submodule from the learning path that intermingles the approach of: small theory step + small practical task + small test. Log what's missing/wrong in my dept log. Give the dept log at the end of the session to the LLM to incorporate/create another test/task.
What I have found useful in this approach is that it will generate a lot of practical tests/tasks for me and it will explain a concept in many ways until I understand it. Also it finds some prerequisites I might miss, but based on my tests and debt log unexpected things I thought I understood surface.
So with the limits of LLM and while building a mental map of the relevant parts it's usually enough to spot the hallucinations, but if you apply structured approaches these are minimal. And it's super good, because the number of practice tests and explanations is endless.
The interfaces are a bit clunky, but current multimodal LLMs are ok with images or even hand writing.
I will recommend that structured approach.
jdalsgaard 17 hours ago [-]
I guess we all learn in different ways. I prefer reading, and can fairly fast find the articles that make sense to me.
Colleagues often suggest podcasts and videos - I very, very rarely listen to them or see them.
The bandwidth is too low. It's not efficient and ultimately I'm bored.
This is a nice project, it looks cute. I watched some of the pages
But I want more than that, more information, and faster - still a Wiki fan.
Also, step number 2 in the flow: have the LLM check itself... Naah, I don't believe that.
But you're not the only using gen ai like that. Take care.
dan_mctree 17 hours ago [-]
Imo the podcast and the video are better served as background material for some other task. The low bandwidth becomes an advantage because it's often ok if you miss out on some parts due to lack of attention.
Indeed it's often a waste of time to just focus on talking people fully if you want to learn fast, reading and especially deliberate practice are better for that. But if you don't have the time, energy or focus, then listening to interviews in the background can be useful supplementally
smnplk 2 hours ago [-]
The other day i was trying to learn why there is no internal structure to electrons and other quarks, but it failed. I still dont know.
pbmonster 2 hours ago [-]
Good news: nobody does.
cautiouscat 18 hours ago [-]
I’ve written a skill that I basically feed what I’m looking to do, some ideas I had for accomplishing it and any other details like tech stack, etc.
The skill then riffs with me, judging my ideas and suggesting alternatives. We go back and forth until something useful comes out of it. This process isn’t unlike how I do normal development.
However, once agreed it breaks the work into “steps”. It then creates a tutorial for me, for those steps, explaining each line, why each change happens etc. I can then ask questions, muse about an alternative idea etc. Then I do the steps, and I’ve learned and gotten what I wanted to get done.
This has been how I’ve been learning Godot and making a game for the past month or so. I didn’t go in blind, I started with a course from GDQuest so I could feel confident guiding the tutorials. I will say though, having a tutor to bounce ideas off of has been really useful.
I still try to figure it out myself, consult the docs, discord etc. But if I’m stumped I’ll run my tutor skill and have some fun.
vorticalbox 17 hours ago [-]
I’ve started doing a similar thing after reading a post on hn about manually applying the code so that you actually understand it.
I have done this for all my work this week and it works quite well.
For one it lets you actually query the LLM as to why, their plans give a high level not every single change and it allows you to correct it as you go and the plan will change.
hasley 8 hours ago [-]
One of the most exciting things about LLMs for me is to have someone with which I can discuss about technical/scientific/engineering things. Since usually I do not have a human at hand (or do not want to annoy people around me).
Especially conversations that involve formulae can get really interesting and insightful.
Of course, you have to be careful with the answers. Especially when the discussions get longer. But usually I see that it is time to stop or to start a new session when the formulae do not make so much sense anymore or when the LLM repeats itself.
But with enough caution, LLMs are really a not-so-bad intellectual sparring partner for discussing ideas and insights.
delifue 12 hours ago [-]
The author says it's "100% accurate and free of hallucinations" but I am sceptical. Although I don't know much about chip production, when I ask LLMs about advanced topics like memory order LLMs tend to hallucinate more than entry-level questions. But the point is that I didn't found that LLM hallucinated before knowing it deeper. It's possible that LLM hallucinates but you don't find out because you are just learning it.
foo42 17 hours ago [-]
I've been using them by reading some docs/wiki/tutorial, then when I think I understand something trying to do a rough explanation to the LLM and ask if I'm right. I'm usually making some analogy to something I already understand a little. I'm usually partially right but missing some key bits at the first pass. I go back and forward asking for explanations of various bits or asking for resources around the area I'm not understanding. Often times just discovering the relevant name for the area of study opens lots of doors. I basically use it like I would talk to a knowledgeable and patient teacher.
As for how useful it is to understand thins, I believe it's still useful and hope it will continue to be.
user- 18 hours ago [-]
The little tool it outputted is nice, but click around the stages and the text is not high quality at all. The snippy titles, abbrievated explanations, I wish a few more iterations and thought was put into the actual main textual content. Especially for 'complex' stuff
aarjaneiro 2 hours ago [-]
These comments make me wonder if people were unable or unwilling to use search engines effectively
Duanemclemore 11 hours ago [-]
Buildings are a fantastically complicated and interesting complex problem space. I found my architecture studio students using it to ask technical and code questions about the buildings they were tasked with designing. I observed that the inaccurate information it was returning was compounding...
To be clear, I say "inaccurate" rather than "wrong" in this case because even if the information it returns is factually correct to the question being asked, students don't have an understanding of the complexity of the interdependent tectonic, regulatory, and spatial / experiential factors of a building sophisticated enough to ask their questions of the specificity and nuance necessary to get a good output that addresses the entire problem.
Anyway - with the students still learning to ask questions the right way, and the conditionally-incorrect facts making their learning more complicated rather than less, I hit on a strategy for them to use LLM's that seemed to help much better.
I suggested that instead of ask the LLM for the factual answer, or even better for the facts and an explanation, that they ask it to direct them to the proper place in the source material to find the answer themselves. Then, to treat it like a lab partner. IE:
Hey Claude I'm looking for "x."
Claude: "look at foo, bar."
Thank you - chapter (foo) part (bar) table (goo) says "car." However I notice that footnote (hoo) says there's an exception if "dar." Which is what I have. Walk me through this exception...
It seemed to have good results as a guide to understanding the disparate bodies of knowledge that they will eventually have to keep together in their heads and work synthetically and non-linearly through, rather than just as an external source of blindly trusted authority.
6 hours ago [-]
mancerayder 16 hours ago [-]
I use it by telling it my background, giving it a rough timeline and asking it to create a learning timeline, save progress along the way, and git push / pull periodically so I can use the same thing on both Linux and Mac. I tell it for each phase in the learning timeline, present me information, then challenge me on it. If it's code, it challenges me with a coding challenge, where I use it in an IDE plugin. If I'm learning something that isn't strictly code, then I ask it to give me info, then challenge me with questions and grill til I get it right. I ask it to save what it think I struggled with, so that later we can drill it again and I can also review it in an .md file.
Does anyone else use Claude like this?
It's sped up my learning by 10x. I struggled with 'just reading a book.' Take kubernetes. I hemmed and hawed and spent years periodically reading some dry book or blog or official doc, falling asleep, and forgetting while I got busy. Now I'm aggressively working with it, almost like I'm addicted to a gamification, of getting through our learning timeline, and I'm excited to move forward as quickly as possible and pass its tests.
It's like a fake teacher, because I can also ask it to drill into a topic or re-explain itself if it made no sense.
The only thing that worries me is, sometimes I'll say something like, "Um, are you sure about that?", and it'll apologize and correct itself. I barely challenged it!
mlpoknbji 17 hours ago [-]
My high hopes were quickly dashed at the step that involved turning a complex topic into rollercoaster tycoon.
meerita 18 hours ago [-]
I use LLMs to learn deep technical concepts. I really like them because I can spend countless hours a day understanding things and building an investigation file with all my findings. I code examples and test the findings. It has helped me understand basically anything.
I'm using LLMs right now to build a terminal browser, a GUI browser, and a PyTorch/LibTorch replacement. It's really fun to be able to learn and make progress this way. It's like reading multiple interactive books, where every concept can be explained again and again until I understand it.
tolerance 17 hours ago [-]
If you're using LLMs to learn or for research, and at some point you don't end up engaging with an actual resource (books, papers, lectures, web pages, etc) then you're playing yourself.
gnarlouse 12 hours ago [-]
Sorta. If its response is grounded in actual material, and you're thorough, it's not so risky. As with all learning, trusting one source is a risk in itself. Hell, I didn't even trust my physics textbooks in college. Physics.
chadcmulligan 12 hours ago [-]
These animations are great. I have learnt so many things from LLM's, cross checking things is easy enough, but the hallucinations are really not much of a thing any more (in my experience). When you deep dive on things it does seem to get very wordy sometimes (this is claude anyway). Its taught me flutter and dart without opening a book (with the occasional reference page), set me straight on monads finally, refined some linear algebra, various bits of history, and philosophy, I'm learning spinors at the moment. Is some of it wrong - maybe, but it's not like my brain is 100% accurate any way, and when I need accuracy I look up references. It is fantastic getting a broad overview of a subject you don't know or a precis of a current subject, and its so much faster.
Hallucinations bring to question what you think you've learned. That's going to cost long-term if you labor under mis-apprehensions until you maybe figure out you learned something wrong.
chadcmulligan 11 hours ago [-]
where's it mentioned on that link? I can't seem to find hallucinations (don't worry found it).
Yes, sometimes its wrong, most times its right, cross checking is fairly easy, not using it because of the possibility its wrong seems a baby with the bath water thing.
I learnt a lot of functional programming from it, stuff I've always wanted to learn, but just didn't have the time and really the sources can be difficult, it really explained things well, and as someone else said in this thread, you can ask questions over and over until you understand, asking a person that (if you can get an expert) would drive them nuts. Maybe my experience isn't typical, its hard to tell, everyone reports something different.
w10-1 17 hours ago [-]
Gamifying the presentation could make topics more accessible to others. For me the overhead wouldn't help with my own learning. Also I've been burned by just learning things mechanistically (e.g., coding, applying algebraic rules), so I'm leery of learning just by making flashcards or models of the topic.
I find LLM's do great for learning when I ask what are the principles, how the main applications work, what are the key drawbacks, where are the growth plates in the field, etc. - the kind of thing a good advisor points to. Sometimes I have to ask it explicitly to use topological order of topics and show relations, which often highlights the gradient changes in the learning curve. For pruning, it's surprisingly good applying philosophical heuristics - Occam's razor, or Derrida's differance (the difference that makes a difference), etc.
And finally, no learning is effective without problem sets, and for those LLM's at times get me over blocking issues.
The degenerate case is memorizing the glib phrases regurgitated back to me; they're helpful and functional enough to get me into real trouble!
praveer13 16 hours ago [-]
I’ve been doing something similar (browser only interactive courses served from GitHub pages) to teach me topics from beginner to advanced
These are high quality at first glance. the visualizations and exercises are helpful.
What is your process for creating these resources?
praveer13 15 hours ago [-]
I just gave a simple prompt to Kimi k3 when it came out, and then just using it and asking it to improve based on my own taste/gaps etc so it’s a long drawn out process. I’m doing that for the first one, will iteratively keep improving it myself as I run through it. Maybe will post on HN after that.
ElijahLynn 18 hours ago [-]
This is a great share, gonna come back to it.
Also worth mentioning that Matt Pocock has a /teach skill that creates interactive, learning sites for learning a new skill.
sathish316 1 hours ago [-]
+1, you can customize it to learn any topic concisely, learn by doing, add challenges, add visual explanations
I am very interested in figuring out how people use LLMs for learning. I definitely have the knowledge, but it is severely autistic in a way.
OTOH, I am curious if there's a "practical value" to this exercise? If the LLM already contains the information and implementation knowledge to implement the networking stack inside an FPGA by itself, what value do I gain by learning about HDL, TCP, the bespoke Xillinx tooling, reading the documentation, reading papers on the implementation and going through every bit of details and theory.
I feel like there's a meta skill that is more worthwhile for "practical value".
mfalcon 39 minutes ago [-]
You can't simplify learning that way, learning is expanding your mental horizon, by learning something you grow and can take bigger things.
throwaw12 9 hours ago [-]
This looks very interesting, something I was also trying to do with my learning.
One thing I wonder is, do you mentally 'fight back' monotonicity of your interactive tool? All seem to be in 3D space, with low-poly, like in a factory moving through the belt and giving you an information + textual description to read more
But sometimes you want to visualize the charts, or graph of simulations, or maybe even the parts of an item in the rocket.
scottrogowski 14 hours ago [-]
The main bottleneck as an engineer is no longer writing or testing code. It is how long it takes to understand complex systems. This is a really nice approach that, if you have the tokens and the patience, feels like I great way to learn something and I think we'll see more and more stuff like this.
I had a similar realization a few months back and am working on a tool that generates "mermaid walkthroughs". It is 1000% less pretty but it is fast and is pretty good at explaining how services work or what a code review does or just as a way for your agent to explain some decision to you.
I wish there was a LLM tool to explore a topic recursively, as a tree or a mindmap. You would start with some high level concept (say "cryptography") then dig further and further to more specific topics.
I think that would be a way more natural way to explore than being stuck on the classic linear output of a LLM.
seabass 15 hours ago [-]
> In plan mode (using CC, or OpenCode) I ask a model to build the foundational knowledge for X topic. I ask it to review the accuracy of the knowledge base it built in the previous step.
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
How do you make that leap?
p0w3n3d 7 hours ago [-]
TBH I was expecting something different. When I use LLM for learning I ask it to make questions for me. It's genius in these tasks. I moderate it to not kill me with difficulty but otherwise it's friendly to help me learn and understand
QuantumNoodle 16 hours ago [-]
> I ask it to review the accuracy of the knowledge base it built in the previous step.
> [I ask it to build an interactive thing]
> I then push it to a new repo and enable GitHub Pages for it.
Congratulations. You are an echo chamber for LLMs. Use it to create, and verify, and post to then be scraped and trained on again.
FailMore 5 hours ago [-]
I would prefer to view code this way - a flow of data between things... a project I'd really like to work on. Has anyone done anything similar?
8-prime 4 hours ago [-]
There have been numerous approaches doing things akin to that. I remember my university working on such a project. Or at least a similar one, representing a code base as a street network with traffic showing how data flows through the system.
I too, would like to have such a tool for viewing larger projects where the flow can be cumbersome to reason about. One issue I guess would be finding the right level of abstraction in the representation.
nirbendavid 9 hours ago [-]
I usually learn compelx topics using the image generator of an LLM, capabilities like GPT-Image2 or Nano-Banana.
I give it like a complex paper -> turn into visualzation or a poster, then ask quesitons and it helps me understand someitmes a very complex paper rather easily. The jump that nanobanana / gpt-image-2 had done is pretty wild.
dwa3592 18 hours ago [-]
i looked at the animations, they look cool, and i don't think i will enjoy learning things that way. as someone else said, there's a lot of content already produced on these topics. i also think the level at which these animations are playing, they are actually hiding the 'complexity' of these topics.
kaycey2022 10 hours ago [-]
That's it? It doesnt feel right. This feels good enough for a casual dinner table conversation and not much more.
AmbroseBierce 13 hours ago [-]
But AI will be better than you at those topics as well, and when someone needs an expert in that topic take a guess who will they approach in such scenario.
I don't think we are even that far when the complexity AI can handle surpasses 99.999% of what humans can handle, where AI make e.g. physics discoveries beyond the grasp of most humans and it will have to "dumb it down" when talking with humans -even physicists- but not with other AIs
reqo 17 hours ago [-]
Something that I have realized recently is that it has become so easy to get an answer to almost any question with the help of chatbots that its almost unnecessary to spend any effort thinking about the problem or the solution. I feel like before when I had to spend time researching a problem to find an answer I learned so many things around the topic itself which helped me understand the problem itself better and gained a deeper understanding. Today it feels like you can have an answer to the most complex questions you might have, yet you gain a superficial understanding of the topic and might forget about it quickly.
blackqueeriroh 11 hours ago [-]
For everyone who thinks you can’t learn with LLMs, Dr. Cat Hicks, psychological scientist and author of the recent book “The Psychology of Software Teams[1],” who worked at Google and founded the Developer Success Lab at Pluralsight, has written two skills called learning-opportunities[2] and learning-goal[3] that use validated learning science to help you learn while using LLMs.
Cat also has an awesome podcast with her wife, Ashley Juavinett, Phd, called Change, Technically.[4]
I encourage everyone to check out her work! She’s dedicated her life to helping software developers get the support they need inside organizations to be seen as humans, not just robots.
It depends what you mean by “learn”. To have a superficial idea of the high level concepts of a step-by-step process, sure, a 3d visualization like the ones this author makes can work. But it stops there.
jchook 9 hours ago [-]
This looks like the most convoluted and token heavy way to consume a bulleted list.
SwtCyber 9 hours ago [-]
One concern though: "100% accurate and free of hallucinations" is doing a lot of work. A second LLM pass can catch some mistakes, but it can also confidently agree with the first one
wiremine 17 hours ago [-]
Personally I'm excited about these sorts of experiments. We all learn in different ways, and these sorts of techniques allow us to create "on-demand" syllabuses and lessons that fit our learning style and learning level.
It's not perfect, but I'm optimistic this will be a useful way to teach/learn in the future.
And, to be clear, I think this will be best utilized within a group/community setting. I don't think it will replace teachers or classrooms.
tuvix 17 hours ago [-]
I don’t think I really agree with the author’s approach here, but I will say LLMs have been a huge help to me as I’ve been reviewing linear algebra and diving into signal processing. Anything in a textbook that I don’t fully grasp or am confused about, I just take a snapshot or copy paste then ask a model to derive it or explain it in different terms.
It reduces friction a ton, but at the end of the day I’m not skipping anything.
afro88 15 hours ago [-]
I'm all for this and I'm keen to try it. I certainly don't want to take away from sharing another neat use case for learning.
But
> What you get is a beautiful animation that is 100% accurate and free of hallucinations
100% free of hallucinations when you're not an expert that can check it is impossible. LLM hallucinations are an unsolved problem.
antonyragleap 8 hours ago [-]
The back-and-forth questioning seems especially useful for exposing gaps in your mental model rather than just producing a summary.
madibo3156 17 hours ago [-]
This guy is severely milking it now. If learning means building an inaccurate and incomplete understanding of the topic then go hog wild. Otherwise https://news.ycombinator.com/item?id=49209049 sums up my feelings about the author's attitude.
mnsh 10 hours ago [-]
having llm audit all my work/code, and generate review pages (as a teacher) of before/after with working examples is incredibly useful. Its something no course can do for me, even a teacher wouldn't have enough patience to go through each one of my mistakes.
having things defined/have correct solution to compare for review is useful, and keeps llm on track. Don't think i would trust llm if it were reviewing it all on its own
0wis 18 hours ago [-]
Really neat idea, I think it is one of the best ways to exploit the combined building and explaining capabilities of LLMs.
I am currently building an app/game to explain friends and family concepts around wealth management and wealth building. Games are a great way to hide complexity while still including it in the « guide » you are making.
sgarland 16 hours ago [-]
The selected topic (chip manufacturing) is being presented at a shallow level at best. Go read Wikipedia’s article [0] and see how deep and broad the topic is. It also skips quite a few steps, and utterly glosses over how insane of an accomplishment EUV is.
This is my biggest societal issue with LLMs: they allow you to think that you’ve “learned” a topic because you read a lot of technical terms. I don’t think it bodes well for the future.
Very cool, I like the visual learning nature of this and the auto play once starting. The game graphics are engaging which counts for a lot these days, I feel my attention span suffering after using agents for the past year.
I've been working on a similar process of pushing to github pages, but focused more on having "practice sessions" with coding blocks to test content. Using webassembly and mock servers to mock backend endpoints Here's one I built to build a full stack llm chat system in the browser.
For background, in 2020 the International Maritime Organization changed fuel standards globally. The intent was to reduce pollution, which is great for human health. But, it turns out some of that pollution was supporting cloud formation and creating a cooling layer globally. And temperatures started to climb when the pollution was removed.
I haven't looked through how it's doing the calculations yet but this kind of visualization is full of possibilities if done right.
And since the details of ship tracks are a newer topic for me, I'm curious about Gell-Mann Amnesia. Next I'm going to apply this to a topic I understand well and will report back. Stay tuned! (will be tomorrow, bookmark or reply to this thread)
zatkin 17 hours ago [-]
I tend to ask the LLM for a single HTML page explanation, with a pedagogical approach. Something about dropping the word pedagogical leads to a more structured outcome, but I haven't quite figured it out why yet.
kurthr 14 hours ago [-]
I am greatly disturbed by the idea that any of the demonstrations involve "complex topics". Whether it was the silicon workflow, LLMs, or EUV, I saw nothing complex. Are these supposed to be freshman undergrad or even high school level explainers? I suppose I didn't see any glaring errors on a quick glance, but definitely lots of details were glossed over.
Perhaps basic special relativity could be done this way, or simple derivatives, but definitely not general relativity or integrals, much less PDEs. I guess I was hoping for a 3B1B type output. Oh well.
kingkawn 14 hours ago [-]
Rejoice for your chance to brag
socalgal2 11 hours ago [-]
Open engine factory, stuff is flowing opposite direction of the conveyor belt it’s on. 100% accurate!
NBJack 18 hours ago [-]
My favorite way to learn infra topics at work right now is asking for a humorous analogy involving monkeys and bananas. I tend to remember the result, and it gives me reference points for new topics.
dcreater 18 hours ago [-]
YouTube has so many truly wonderful videos on chip production. I admire your approach but it seems like a lot of people are in this ai maxxing phase where they reach for ai for everything despite their being ready, high quality things already available for free
AlotOfReading 18 hours ago [-]
I'm also not sure you can really learn chip production from widely available public information. It's a hugely complex industry where the details tend to shape larger strategies. For example, you can't really understand the relationship between Micron and TSMC without some awareness of the trade-offs of memory processes for peripheral transistors.
tayo42 18 hours ago [-]
YouTube is just a big dump of information. Having a structured way to learning along with interacting helps you learn. Otherwise you're just binging information
ultra_nick 18 hours ago [-]
Oh, I do similar, but using single page websites.
Last month, I read The Prince and had it make a text adventure campaign for me.
For a lot of other topics, I often just ask it to create a simple python example that I can run.
shevy-java 2 hours ago [-]
Looks like a horrible flow. I feel sadness for the guy.
SalaevAl 18 hours ago [-]
LLMs can help you understand a language, but they can't replace learning the vocabulary. Words and phrases still need to be learned the old-fashioned way: repetition
CrzyLngPwd 17 hours ago [-]
AKA - How I skim read a lot and learn nothing of value.
tamdnq 12 hours ago [-]
haha, I did the same to study new topic and then realized how personalized education will turn out in the near future. The only thing left is keeping a curious mind and ask a lot of questions !!
scotty79 3 hours ago [-]
What I'm missing is an agent window near the text that I could immediately inquire about the details of the process.
IshKebab 18 hours ago [-]
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
How do you know if you're learning this for the first time? Very risky to learn from LLMs. I've done it, but you have to keep your wits about you. Lots of "oh of course you're right - what I just told you was completely wrong".
internetguy 10 hours ago [-]
this is pretty cool! it feels very good for getting a brief summary/overview of something and getting a beginner's level grasp.
throwaway63467 17 hours ago [-]
That seems like a terrible way to learn. It’s a neat animation but cmon, there’s like educational TV programs from the 80s that explain this so well, in Germany there’s “Sendung mit der Maus”, not sure if they have a segment on chip manufacturing. But in these clips you can at least see the real stuff instead of some half wrong animation, just let the LLM write a few paragraphs for you or better find an ACM article or book on the subject, probably still takes less time than coming up with that animation…
God does everything have to be productized and glorified as if you’ve invented a new way of learning. Read some books!
novren 16 hours ago [-]
Building learning artefacts is under appreciated.
Inthink this was always the best way to learn. But it used to require immense work for a teacher.
My compiler course is a great example - program in plug in a stage of a compiler.
These exercises can be made on demand and incredibly easy now.
kannanpoem1984 6 hours ago [-]
This is cool project !
curtisblaine 4 hours ago [-]
Those animations are... series of steps, each with a description? How different is this from a plain text list? It's not that you can see forks or that the layout of the assembly line gives you more information.
If anything, a plain text list is easier to scroll backwards and forward?
hathym 18 hours ago [-]
Too over engineered, few text paragraphs would suffice to understand the topic.
shay_ker 14 hours ago [-]
i've asked llms to write a presentation for me on a given topic. for whatever reason, that finds the hidden layer responsible for explaining things well.
TormentNexusAI 14 hours ago [-]
The biggest win for AI dev efficiency is cutting down what gets loaded into context. Semantically matching tasks to the top tools helps a lot.
fosterfriends 18 hours ago [-]
Is there a way we could turn this into a reusable skill?
light_hue_1 15 hours ago [-]
This animation is worse than useless. I've taught a lot of students. This is not learning, it's stamp collecting.
You're just memorizing a nonsensical recipe. What are the constraints? Why do we do X rather than Y? How does a particular thing scale? etc.
All you're doing is fooling yourself into thinking that you've acquired some knowledge. When in reality you haven't even learned the basic mental model to reason about this stuff.
You've learned something when you have a mental model that makes correct predictions. Until then you've memorized it at best, and as with most memorized things it will decay exponentially and will be gone from your memory soon enough.
17 hours ago [-]
jb1991 16 hours ago [-]
anyone using LLM to learn a new language? got any useful prompts for this?
federico-peconi 17 hours ago [-]
not all knowledge can easily be translated into a pipeline alike game imho
zapataband1 17 hours ago [-]
"What you get is a beautiful animation that is 100% accurate and free of hallucinations."
Do you know it is free of hallucinations because you crossed checked it with the source material or because you told the LLM "don't hallucinate"
spacedcowboy 18 hours ago [-]
Another, possibly easier, way is to ask an LLM to give you a quiz on a topic, and then discuss your answers with it.
Surprisingly effective.
chasd00 18 hours ago [-]
I had an internal company assessment I needed to pass before end of our fiscal year. The study material consisted of 10 ppt decks about 80 slides each (so around 800 total). I had an AI read all the decks and compose a study guide with quizzes along the way. It came up with a 100page word doc that I used in place of the decks to prepare. It worked very well for this including, like you said, quizzing me over various sections.
(Yes I confirmed it was ok to use AI with the material)
weedfroglozenge 8 hours ago [-]
The self importance this author has is sickening. No wonder he's chronically talking to chatbots.
fosterfriends 18 hours ago [-]
I love this idea!
wyzer 15 hours ago [-]
Kids use agents for exam training, rather than cheating. Which is great as long as the LLM does not hallucinate
epolanski 17 hours ago [-]
> I personally find the style used by LLMs to explain things difficult to follow. It's just too simplistic
I also struggle with LLMs explaining things, but for the opposite reason.
I consistently have problems to get short, precise but plain/simple answers.
Instead I'm overwhelmed with walls of texts, often filled with jargon that is a mixture of imprecise and unneeded.
The style at which I learn better is by asking about stuff interactively. I ask you what something is, you give me a 3-4 sentences top answer. Then I explore and dig into the topic from your answer on the things I want to know better.
UltraSane 17 hours ago [-]
I recently used Opus to create a learning plan in ClickUp for every Tailscale feature and it worked very well.
eager_learner 14 hours ago [-]
i had Bolt create an interactive app teaching me Oberon language but despite several attempts to fix issues, there were still glitches in unexpected places.
So the experience was... meh.
Then I went back to a book written by humans (Eric Nikitin's "Realm of Oberon").
nalekberov 7 hours ago [-]
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
I am pretty sure this person is
a. Ignorant of the fact, that AI is designed to give just answers, they do not care whether they’re true or not
b. Have never Cross-checked Information given by AI with reliable sources
aswegs8 18 hours ago [-]
I mean, yeah, fun project. But that whole game could be like 15 mins of prompting.
I'd be extremely cautious to ask it to have it explain any specialized concept even from a document.
In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.
Here's how I do it: I open Baby Rudin (3rd ed.), second chapter, and read the main text - absolute brutality. I unpack almost every sentence with Claude/GPT until I finally get what's going on. No ELI5 nonsense, just examples and counterexamples galore while absorbing the techniques and the way of thinking in analysis/topology. How do I know I've learned the material? By solving every single problem in that chapter. Here's the thing, though: the problems in Rudin can be brutal and decoupled from what's in the text, so if you can handle them, you've definitely mastered the material. No 500-word summary of analysis here.
If you actually want to learn analysis, there are a nearly infinite number of friendlier resources (e.g., Understanding Analysis by Abbott).
Learning to unpack difficult text on one’s own is a valuable skill. Research papers often require a similar amount of suffering, and at the frontier of knowledge, despite all the advances we’ve seen, LLMs seem to have absolutely no understanding or intuition. They are much better at things that have been expounded at length by humans before.
I would argue that if you’re going to use an LLM to make Rudin easier to understand, you are not learning how to absorb difficult material, nor are you learning analysis efficiently.
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory.“
-Socrates, on writing. From Phaedrus
Secondary source: https://www.historyofinformation.com/detail.php?id=3439
"For the correct analogy for the mind is not a vessel that needs filling, but wood that needs igniting - no more - and then it motivates one towards originality and instils the desire for truth. Suppose someone were to go and ask his neighbours for fire and find a substantial blaze there, and just stay there continually warming himself: that is no different from someone who goes to someone else to get some of his rationality, and fails to realize that he ought to ignite his innate flame, his own intellect, but is happy to sit entranced by the lecture, and the words trigger only associative thinking and bring, as it were, only a flush to his cheeks and a glow to his limbs; but he has not dispelled or dispersed, in the warm light of philosophy, the internal dank gloom of his mind."
-Plutarch, on listening https://books.google.com/books?id=0U-hsAonP1AC&lpg=PA50&dq=p...
I didn't read the whole passage, but it seems to be talking about the same thing.
A decade ago, a google search for study guides written by another professor would have been slightly slower. A decade before that, you'd be even slower fumbling through several books. Every single word could at least be trusted. You don't get that from an LLM.
The fundamental service a teacher provides is personalized feedback, quickly identifying where you are stuck and focusing the explanations and exercises on that area, drastically increasing the speed and quality of learning versus the self-supervised route.
The lack of this closed loop effectively killed the high hopes that were placed in e-learning and MOOCs 15-20 years ago, TV learning in the 1960s and many other failed revolutions, seems every generation has its own version.
It appears to me LLMs have a real potential to close this loop and become the failed educational revolution of our own generation.
This has been a huge blocker when I tried to study advanced math myself. Many of the exercise books don't have worked out answers, so often you're either stuck or you have to hunt a variety of sources online for solutions and advice. It kills flow.
It is also significantly more engaging and fun.
The scientific basis for this statement is unclear.
In [0], I ask: „When applying Hidden Markov Models to POS tagging in NLP, what do the latent states and observations usually represent?” I then follow up with some specific questions and requests for walkthrough. You can’t see it from this conversations, but I have Wikipedia and a bunch of other resources open in separate pages, cross-reading, and I follow up with a handwritten toy implementation of a Viterbi-based POS tagger once my mental model crystallizes. This is very different from a 500-word summary of quantum physics, and I still had to put in effort (this is unescapable!), but I found the experience rewarding. Also note that this is relearning of a topic that was part of my uni curriculum but long forgotten.
In [1] and [2], I’m learning Spanish by reading García Lorca’s poems. Here again I’m going through the texts with a dictionary, and augmenting my learning with what a dictionary won’t tell me: given the usage of a word or phrase in this specific poem, is it something that could occur in everyday speech, or is it poetical?
[0]: https://chatgpt.com/share/6a743bc7-d0dc-83eb-acc9-8f2faaffc4...
[1]: https://chatgpt.com/share/6a731be6-26bc-83eb-8d21-c965da5364...
[2]: https://chatgpt.com/share/6a731c01-5604-83eb-a5b8-cd1295d0ef...
That's what you'd get if you asked a leading physicist too, so are you saying LLMs have achieved human-level intelligence?
- Getting through textbooks and lecture notes. LLMs have gotten very good at answering basic questions on quite advanced material (e.g. representation theory and quantum field theory). By asking a very specific question or even giving the LLM a screenshot, I can get unstuck a lot faster.
- Learning e.g. new python packages. Instead of hunting for examples on Stack Exchange, now I ask an LLM to write a minimal working example and then build off of that. By writing most of the remaining code myself and only using the LLM to answer questions, I've been able to learn new packages significantly faster.
In both cases, the LLM isn't providing the curriculum or guiding what I learn. The textbooks, papers and coding tasks are. But now I can pick these things up much more efficiently.
The other day I realised I had no idea how DNA and life works. I guess I studied it at high school (25 years ago), but maybe it didn't go into much detail or it just didn't click.
So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
Am I going to become a biologist and study the origins of life from that? Definatley not! But if my kids need help on their biology homework, I now understand the basics of it.
If you're only checking your understanding against the one source you used to obtain it, how can you tell whether your understanding coincides with reality (or rather, with general scientific understanding), and not just with the source you read? And I'm not asking just about ChatGPT; the same question could apply to any source. Books are not exempt from containing errors.
Turns out that the author had done a thought experiment but neglected to factor in the rotational inertia.
Your personal assertion is quite wrong at a fundamental level. Biology refers to the field or study, not individual specimens. The field is comprised of the understanding that people over time compiled on nature, along with arbitrary frameworks that help people organize and reason about the topic.
Take the concept of species, and species classification. A specimens exists regardless of being classified or not. However, the same specimen can be classified differently depending on the state of the body of knowledge at the time. In some cases you had species being reclassified due to new findings, such as genetic tests.
Once you understand this, you learn that you can't claim that a field of study is anything other than abstractions and partial and incomplete observations compiled form people throughout time.
In any case I think you have an overly narrow definition of learning that we're unlikely to come to terms over.
Those boundaries are completely imaginary and don't exist in reality. In reality there's no biology, there's only elementary particles interacting physically. Whether you agree with someone else to classify a phenomenon as biological or chemical, you're not refining your knowledge of the real world, you're just performing an organizational task.
But, say, how many times the tympanic ear evolved independently is a real phenomenon that can be investigated, and it's something that you're either correct, incorrect, or ignorant about. If I tell you it evolved five times, what more can either of us gleam about the real facts by just discussing this factoid back and forth, if neither of us has access to any additional knowledge or way to put this datum to the test?
The equivalent for biology would be to grow a plant or a few plants and animals successfully. That's operational at a certain level, you could also be operational at a lower or higher level.
EDIT: Perhaps not the best example, because the size and shape of the Earth are data that are repeated often enough that an LLM would be unlikely to quote it grossly incorrectly, but I think my point still comes across.
So, I think the point OP is making is that most people don't really check sources while learning things 'the conventional way'.
If you're only ingesting information from a single source, be that a book, a teacher, or an LLM, then you haven't really learned, because your knowledge base is unmoored. You can't learn history by studying Tolkien's mythos.
EDIT:
I think it's a kind thing you need to tune yourself into. OTOH, I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it ¯\_(ツ)_/¯.
What do you mean by "successful at life" here? Genuine happiness, fulfillment in life? Or in the sense of doing well by what society holds as it's current interpretion of what one should strive for, and otherwise just kinda drifting through life?
Because if it's the latter, I'd say that is to be expected. It's much simpler to put your energy into fulfilling the expectations of whoever is your superior in your current group, mostly get the expected reward, and then just coast. Reflection and experimentation, which is required to get to self-consistent views, takes effort and and the willingness to question existing beliefs, which will also be uncomfortable times.
Exactly. This tells you where something is off. The problem may be your lack of understanding or wrong understanding, or it may be with the source, or the framing, or you may have hit a genuine lack of data - still, the puzzles don't fit in some area.
And yes, not all self-consistent understanding is correct. But all inconsistent understanding is incorrect. And the more knowledge you gain, the less likely it is that it'll all connect self-consistently, but still be very wrong.
Aka confirmation bias.
We like explanations that fit what we expect, even if they're completely wrong.
A particularly bad example is higher maths - a wiki pages on a complex mathematics topic often reads like "A gruncheon is a worch in the brashation of plusters" and each of these words is a separate page or topic. Of course, you _can_ in theory 'just' click through all the tree of linked pages to understand a concept ...
For DNA the page (scanning it now) is well laid out, with images (including a spinning Rasmol? image) and lots of detail. However, the detail could be a drag on understanding for some : There are 'nucleotides' and 'nucleosides' and 'nucleobases'? There are non-canonical bases? Supercoiling? Z-DNA? While I know (most) of these things, it is because I've learned about them in other contexts, or by direct instruction.
I'm not saying it is impossible to understand DNA from that page, but it is likely to be harder (for some?) than a more conversational approach to learning.
I just asked Opus to "explain DNA to me in simple language" and the two are not even in the same league in terms of quality.
I mean - it is certainly better ... but it is still a lot of stuff. For example:
> Part of an organism's DNA is "non-coding DNA" sequences. They do not code for protein sequences. Some noncoding DNA is transcribed into non-coding RNA molecules, such as transfer RNA, ribosomal RNA, and regulatory RNAs.
Do you _need_ to know about tRNA, rRNA, and operons (?) to understand DNA? The thing about an encyclopedia/wiki entry is that it has to cover the whole topic. This is a strength for reference, as you can scan it and find the bit you need. For learning from scratch, I can see that a conversational approach (with a human or LLM) has advantages where the learner can direct the level of detail and path through the material.
Ultimately, both are worthwhile, but I can also see the strengths/weaknesses of both ways to learn.
That can help a bit.
I am studying for a Masters degree in Computer Science with AI and the lecture notes are like Wikipedia sometimes. Incomplete, perhaps assume pre-knowledge that lots of Masters students won't have. All of these I have taken to ChatGPT and got great explanations, diagrams, graphs etc.
This is very simple to validate and verify. You could argue it may find false primary sources.
You can condemn models for a variety of other things, but acting as if this is still reality shows a lack of understanding as to modern model capabilities
> Cause if you went back and forth with ChatGPT for an hour it definitely hallucinated and lied to you at some point.
If you're asserting that this is not the case today then that's going to be require pretty extraordinary evidence. "Chatbots use Google now" is not evidence that the information they provide is in fact correct.
They don't hallucinate all the time like they used to, no, but I'd be very surprised if the majority of these sorts of conversations were free of major factual errors.
I frequently notice degradation in the model model's ability to remain coherent when it searches for information online. For example I might ask Sonnet 5 "how do I build a shed" and during its search it presumably comes across an article which talks about building a shed out of paper mache, then the model responds with something like "I caution you against your plan to build a shed out of paper mache" -- Wait, what? Who said anything about building it out of paper mache?
Its easy enough to prompt ChatGPT for primary sources when doing research to validate any claims its making.
1) there's a good chance it was subtly misleading (or just wrong)
2) you probably won't ever use the information in any meaningful way and will likely forget all relevant details in a few days
3) you almost certainly could have spent that time better actually doing or creating something - actually doing real learning and making real progress
Once I saw that I realised just how padded the content really was and started looking elsewhere.
Other good tells the content is low quality or a bad fit for video are excessive amounts of talking head (should have been a podcast) or stock video clips (it's video, show me something).
But this was also the joke about TED Talks: everyone came away smugly confident they were part of an amazing tide of progress, ignoring any engineering or scaling challenges.
https://github.com/mattpocock/skills/tree/main/skills/produc...
The point is not that I’m learning Concurrency in a better way using LLMs, it’s that I can apply this style of learning using bite-sized, hands-on, visual explanations, quiz to any topic in the future. The lessons it generates are just code, and you can ask it to type check the examples, validate with recent libraries, use analogies to learn something better.
Try it before thinking it’s just ELI5 or Summarization or assuming it’ll be hallucinating without verifying facts.
I've spent a lot of time with LLMs for the last two years. Something I've tried, almost for decades, is to learn enough CUDA programming to be productive with it when needed. About 6 months ago, after again banging my head against it for weeks, something finally clicked and I feel like I've overcome the initial step of at least grokking the needed ideas so I know where to go next, and I can actually write + compile + use kernels made for my use cases. I won't claim to understand everything, but I couldn't do what I can today, before I learnt the things I now know.
~2 years ago, because of my very weak math foundation, I basically said "Well, CUDA looks really interesting and really fun, but it's too difficult, lets focus on other things", even after reading some starting resources and stuff. But, by asking countless of dumb questions to LLMs, forcing it to steer me in the right direction, when I'm otherwise just driving on the highway or what not, I finally feel like I have a grasp on something I earlier only dreamed about understanding, and I'm able to be productive with it now.
Stuff like Triton, nvidia warp (the language), numba, cupy jax/pallas and so many others really paved the way. You can start out really high-level, run a profiler and then dive deep into the bottlenecks.
TL,DR: Keep going, it's a great time to have fun with GPUs.
Well, yeah, but what I've being doing is learning proper CUDA, not "Python-compiled-to-CUDA" (otherwise it'd take like a just a week to understand enough :P ) and that's looking more or less the same today (although bunch of more complicated stuff piled on top of the fundamentals) as it used to, AFAIK.
With that said, the environment is a lot simpler to setup today at least :)
I believe more and more production code is running kernels which didn't originate from the traditional cuda cpp route.
I wouldn't say one is dumbed down either, just different, at least the entrypoints and how you end up using the different solutions.
I'm currently experimenting with cuda-oxide for some new simulations, and managed to keep the entire simulation within just Rust essentially, while going the "traditional" (maybe better term than "proper"?) way I've ended up with a bunch of .cu files and then integrating them (via cudarc usually). Kernels themselves feel the same across both, but the integration clearly makes them different enough that I think it's worth distinguishing them, at least for clarity if nothing else.
If someone else already knew Rust but not C++, wanted to get into CUDA programming, going the cuda-oxide route would probably be easier and more familiar, than cudarc, I'd guess. Personally I'm not sure what route I prefer yet, both (as always?) have tradeoffs.
Luckily I had my dad available who is a scientist, during my high school and he corrected several fundamental mis-explanations of my teacher that even to my mind logically simply didn't add up. I learned not to relay this back to the teacher of course and sometimes regurgitated the wrong answer in tests. Not everyone is so lucky. This has been my frustration quite often. Textbooks are sometimes wrong, both because the author really doesn't have good expertise on that slice of the topic or out of didactic simplification reasons. Having an LLM that can consult the real grownup literature and give the full story, not the birds-and-bees is quite useful.
Of course "it's just simplified" is a defense that can be attempted to explain away all mistakes.
Dunno, I've been learning a lot of Rust in the past few days. Just dove right into a project and asked AI to teach me stuff on a need to know basis. I'm actually getting used to Rust by now.
It takes way more time to master and be very comfortable with a language due to its ecosystem, though. Some languages are more likely to click with a person, yet underneath they are all the same (minus the functional languages that form their own group), e.g. some performance issues may require looking at the generate assembly code.
However, saying you can't learn "anything" is just too strong. I'm definitely managing to distill the AI's weights into my own brain.
Additionally I asked it to also give me problems relevant in my business domain so that I learn how to directly apply the knowledge in a realistic scenario.
I am getting much more comfortable writing rust than I was barely two weeks ago. More than I was just reading tutorials.
The truth, as always, is in between. There’s loads of people using it for useful things, learning, automation and getting good results. But it’s also wrong enough that you need to deploy it carefully sometimes.
Don’t worry about either group. Keep objective and use AI where it helps and do it yourself where you are better. That’s all.
> I want you to create a tutorial series about X for me. The prime objective is that I improve in topic X so never provide a solution but guide and teach. (for programming never write code). First create a question catalog to assess my current level.
Then I would ask it to structure the tutorial challenges in the following way: - Goal - Concept - Instructions
I figured that if I don't need to read any additional material on the topic the LLM is giving me too much information and I need to change the prompt. Works for me and I used this too learn topics I feel now comfortable with, like nushell, opencyper, elisp, boot loaders etc. But maybe you don't consider this "complex"
From the hilarious "There are 3 ways to learn. (Knowledge Fight Animated)"
https://www.youtube.com/watch?v=NsqZZiWDHAQ
If today's top LLMs are reliable enough (without grounding) to academically learn "complex topics" from, may be I need to adjust my priors. I must say, I do find myself chatting about other topics (without the need for grounding) that I'm trying to "absorb" (not really learn), like Behavioural Psychology & Philosophy.
[0] Products like NotebookLM are built specifically for such usecases.
LLMs give me structure based on my current skill level. And basically always I accompany this with books, I love reading. It's a nice combination for me.
You literally proved the OP's point ... thinking you're learning. More like scratching the surface, with lots of invalid data while not being able to recognize what's invalid.
It's like with latest vector of attacks being spamming Github with malware injected in proper looking code in hope of AI to index it.
Then you paste the code because you don't understand it, but you take it as working and only doing what you've asked for.
What level of evidence would be sufficient for you to accept that a model may be able to teach a concept?
I'm happy to take on this challenge with a topic of your choosing, but I don't believe there will be an evidence base that satisfies you that the knowledge is earned or deep enough.
It's about guiding me in _doing_ exercises so I learn and I can evaluate if I learned something if I can apply the learning myself.
One thing I've begun to notice is that LLMs list of a bunch of interesting stuff and raise all these thing you consider but often sometimes you just want a more focused response, so these scatter responses kind of lead you to being overwhelmed and losing focus on what you really wanted to do. At least I've started noticing this.
Like I'd ask about some statical approach taken in a paper and suddenly i'm being bombard with all these potential pivots and things I really need to consider, I kind of just want to consider 1 thing at a time and come to things once I fixed the immediate issue. Sure I have no doubt these other pieces of information are useful but it's just not the most useful information I need right now.
This is less of a problem with coding agents more so putting learning related questions to an LLM, like is the method covered in this paper, yes no? instead I get an exhaustive but overwhelming and indirect response that contains part of the answer. I just wanted to know if it was worth my time going through the paper but now I'm being bombarded told all this tangential information, which is unclear to me if I need to consider right at this moment, it's really distracting.
Maybe this is something others adapted to but i've resorted speaking past it saying, "this is the question please stay on topic" or literally "one thing at a time please" and then it narrows in, but they really stretch your attention thin if you're not more aggressive with keeping them on topic. The smarter models are better, and if you use max compute it does a better job.
I think they can definitely be helpful for learning, but you got play an active role, you can't just consume what it says like content.
As a tangent, I think the concept of pop science has wasted so much time of what could be considered brilliant minds. I can't believe how much YouTube people I consider really smart consume under the guise of "learning stuff." And the videos are always designed to be addicting and to entice you to watch other of their stuff, which makes sense, because theyre a business, not a school.
I'm guilty of wasting time on YouTube as much as anyone else (I like watching stand-up routines and Red Bull extreme sports) but I am never under the guise that I'm doing anything productive with my time. Its okay to have fun learning, but I always felt that entertainment and education should be kept separate. You gotta learn something intentionally, not just get it served to you via algorithm.
Note im talking about the educational "shorts" not the 60+ minute deep dives that are basically a college level lecture.
It's the kind of high-brow entertainment that makes you feel like you learn something.
For me, most "push" things are edutainment, whether that'd be Youtube videos or public-broadcaster television programs. Things you seek out yourself are not.
In a similar vein, there's "newstertainment" (news that makes you feel like it's important to watch, but actually changes nothing tangible about your life).
On the other hand, I’m fine with not being an expert on topics outside of my domain, as long as I retain some basic knowledge and fun party facts. So there’s that.
Also learning is not just about truth, it is about curiosity as well. The pop-sci metaphors could actually good for satisfying the curiosity of let's say a 10-year old. What to learn and how to learn is ultimately at the judgement of the learner. The better the judgement, the more the learner can stay closer to the exact scientific details.
fable taught me about the beta binomial and large observational studies that came down on both sides of the question about whether per family births are truly binomial. it also told me about countries like the uk and uae that are inching towards national genetic registries that might answer such questions definitively in time. as well as the efforts in Cyprus in this 80s to reduce beta thalassemia through voluntary testing of couples pre marriage.
I really enjoyed and learn a lot of things from Karpathy's and Andrew Ng's video. sure many don't really teach you anything, but I could say many others are useful too. Maybe it depends on the way we're prompting as well? it seems useful for some like Terence's message that was shared few weeks ago
https://news.ycombinator.com/item?id=49010345
"use Socratic method"
For some things you still need videos and practice but cmon, I don't get this generalised hate on LLMs, they are based on what us human wrote anyway.
You wrote a wall of text just to say you struggle with learning when using some media. That's fine, each one of us struggles with different things. However, I hardly think it's fair to extrapolate your personal struggles with learning styles to everyone in such a sweeping approach, or that this is relevant to the topic.
If you want to go back to the basics, LLMs in the very least work as chatbots that you can use to follow the Socratic method to guide your way through your learning journey. If you still struggle with learning when asking questions and getting specific answers to them then it's safe to say LLMs are not a factor.
And then I begin to think to myself that I should just read a book on the topic written by a trusted source who put a lot of effort into teaching the topic properly and presenting the information in a thoughtful way. So, I am back to books and mostly try to use LLMs to clarify certain questions or ideas I have.
For example having an LLM summarize a dense topic and to find books so that you can filter faster and spend time reading those books works way better than having the LLM summarize the books or the topic (or even relying on second hand information). Another one is having the LLM quiz you on your topics of interest. With questions tailored to attack specific areas that you struggle with. Its wonderful at this, nothing I've used comes close to what an LLM can do here.
You define for yourself what your goals are, slowly refining them as you learn more, and use LLM as a tool. This ,I find works best for learning.
It's long been the case that the best way to learn something is to teach something.
Which is pretty unfortunate for those that want to learn. I used to enjoy writing documentation at work, it was my favorite part of the job. And it did feel like it benefited me more than it benefited all the people that were (or weren't) reading my documentation. Now I can't really justify spending much time on docmentation when LLM's can do it in a fraction of the time and it's "good enough"
Is it perfect? No. But it suffices most of the time in a pinch.
Can’t speak to the nuke industry, but it’s pretty good at aviation related things.
Then elaborate in small chunks so the human domain expert can remain focused during review.
This is a great idea. I'm going to try it.
And you notice when it's a topic you know well or something like software where you can immediately tell the options it's giving you don't exist on the page. Leading to the amusing statement "LLMs are bad at what I do but great at everything else".
The question: what's the net positive gain of turning people who know nothing in a given field into sub-novices, while weighing actual experts down with work slop and marginal returns?
And I wonder what the true cost is of arming so many novices with that level of dangerous knowledge.
Tangentially, but related: I'm old enough to remember when the spirit of your comment was pervasive on HN.
I’m old enough that I worked my first IT summer job the same year slashdot was founded. I’ve seen a lot of tech tribalism form and dissipate, and this one didn’t feel organic. My gut says a lot of the us-vs-them tension originated in a deliberate campaign to cast AI boosters as the tech industry in-crowd, and ‘other’ the people not on-board. Who knows.
E.g. the entire framing to combat complaints about shortcomings was, "It's not the tech. It's you. You're just not doing it right. Wrong setup, wrong workflow, add this to your .MD, use loops, etc". Every complaint was immediately met with this same treatment by a swarm of vague bro-bots that materialized from the ether. The core message? Always the human's fault.
And, don't get me started on the waves of newly minted expert AI creators, making recommendations without showing a single example of what they'd supposedly built.
I'm sure some bandwagon organic creators tried to cash in on the genre, but encouraging that was also part of the point.
Just hoping folks don’t get hurt due to people not understanding what they’re doing with these things but believing they’re competent.
The body of literature on learning theory, and beyond that on specific types of learning and specific mediums such as learning from text is so rich there are way more useful models to draw from. Believe it or not, prellm, researchers in the textual learning field had already demonstrated you can achieve performance equal or better than novice tutors using pretty basic computer aids that follow specific hint/pump interaction structures. Guiding an LLM to use these findings has evidence backing it and is way better than telling it "i guess be like socrates". The problem is, to realize there might be richer more effective and highly researched ways of tackling the problem beyond the first fart of a thought you had one afternoon requires the deep respect for expertise and specialization that precisely basically everyone in the AI space right now fundamentally lacks.
You need much more time and guidance.
TLDR: Actual human tutoring sessions were recorded and analyzed and Socratic questioning was barely used at all. Instead the following pattern was observed:
Pump — "Uh huh?" "What else?" Costs nothing, so try it first.
Hint — points at the region of the answer. "What about the pumpkin's motion sideways?"
Prompt — fishes for one specific word, with the sentence frame supplied. "The pumpkin keeps moving forward at the same ___?"
Assertion — just says it. "It keeps the runner's horizontal velocity."
This statement would out you as someone who didn't attend an elite school.
https://adaptive.bounded.cc
Trying to diagrams/animations didn't yield good results even with frontier models. But pure text, any model does a decent job.
So it may be very slow or become unavailable, back end can't handle that, no caching whatsoever.
I'd imagine an application that uses LLMs will be created that better manages learning. It's just not clear what that UX is yet- it's obviously not just a chatbot
i run into context window limits, or practical limitations of digitizing the book
I even wonder if this behavior is due to next-token prediction architectures, somehow.
I know you probably don't consider it dense but wondering if someone can shed insight.
I find them like empty calories, like programming youtube tutorials. They maximize for feeling learnt instead of steady progress
human conceptual thinking is very much a multi-dimensional graph, which relies on light "approximate" concepts that are "good enough". LLM AR token generation is extremely one dimensional and doesnt care about the "weight" of the concept behind a token.
LLMs hold billions of parameters in "mind" at once. humans hold like four "concepts".
This is the essential mismatch and the primary reason LLM conversation can be so painful and exhausting.
Explaining this and limiting "concepts" to four at a time tops is one of the very few AGENTS.md / system prompts I always use, and it has proven invaluable time and again.
Thinking traces show how effective this is at forcing the LLM to simplify its thinking.
[edit] Also, myself and nearly all of my peers are struggling to choke down the flaws of LLM tooling along with the benefits. the speed at which LLM adoption is being forced, without truly crafting them into quality tools first, is not ok, and not normal.
LLMs have stirred an inhumane hunger and fear. the tech is fine, but the way tech companies (creators and consumers) are behaving should be deeply questioned.
it's NOT normal. it's not ok.
I just want to understand more.
Also, would you be willing to share the actual text of it that you put in AGENTS.md?
The full PDF is worth a read (Figure 1 may be of interest to many here): https://www.cambridge.org/core/services/aop-cambridge-core/c...
If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
Personally, I find that its generated prose tends to have an undue weight to it, almost as if every topic I ask about somehow bears a heavy burden, or is otherwise load-bearing, to use its parlance.
Quite puzzling, really.
I think this is one reason why LLM text is pretty exhausting to read for long stretches.
It's possible that this quality you describe stems from the extensive training corpora utilized by the major AI labs. These almost certainly include work from the esteemed economist Jacob Silj:
https://www.youtube.com/watch?v=Poc1upTejD8
I have a personal theory: LLMs are *fundamentally* handicapped at perceiving what's going on in the mind of the human (this can't be "innovated away") and that's at the root of what makes them suck at conversation.
Next time you're chatting with someone, notice how much understanding is shared without anything being said. E.g. the other person might share something deeply disappointing, and they can tell without you even saying anything whether you get what they're going through. This unspoken-yet-communicated information guides the conversation. Or as another example: humans can read the room -- you walk into a room and immediately adjust your demeanor based on what you see and sense.
LLMs are totally blind to things like this, and this adds an inescapable awkwardness to interacting with them. I don't believe they'll ever grow out of this. Which thankfully implies more long term demand for humans instead of robots. :)
Does anyone have a read on if this is primarily a Claude issue, or if all LLMs do this?
I view LLMs in education similarly to office hours. Some people abuse it to get homework answers without grappling with the material, but the optimal amount is not zero.
LLM certainly not a replacement for a book, where you get someone’s extended personal approach to a topic, thoughtfully organized, reviewed and edited, often times actual courses taught based on it, with answers checked and errata available online.
Perhaps the best example has been a native macOS app that is a completely custom text editor with built-in debugger, lsp support, fuzzy finder, etc stuff you'd expect. Inside the same app is a library of books i can read within the app completely formatted and for every chapter/section of each book that is a quiz to take (LLM generated of course), a "recitation" tab where i am asked a question and say outloud my response to the AI to evaluate me on and then finally practice problems to do within the custom text editor (these are usually programming books). The reader also has ai re-write built in.
As neat as this is, and i worked through K&R like this, i have ultimately fallen back on "just read the damn book and go to the AI when you've got questions."
> I get exhausted reading LLM prose
So much this! If I see one more sentence with the words "genuinely" juxtaposed with "load bearing" my head is going to explode!
btw, I am building the tutorial here for anybody interested in this topic: https://github.com/avilay/learn-probml
it also researched vision correcting displays for me and i can finally put that idea to bed - i was never really going to pick up an optometry textbook tbh. plus it was able to pull together a bunch of geometric and physical context about light and the eye plugging exactly my personal knowledge gaps.
in general i suspect these materials might not be that interesting to others because they are so custom to my learning style and personal needs and preferences.
these are usually not one shot documents but rather many prompts deep before i get something I’m willing to sit down and read or study. but dramatically quicker than assembling it myself from primary sources. i wouldn’t say it matches master expositors but then they’re not available to write on any topic i happen to need right now.
plus I’ll just have a live voice discussion with the system when i go for a walk and there are still things bothering me on a topic. it takes a little patience but if i’m in the mood it’s amazing.
i generally find that it can help track down specific references if i suspect hallucinations. but especially on factual topics my experience so far has been extremely encouraging.
So I have the LLM offer a very short explanation of something, and from there's it's just me asking questions. Anything that feels fuzzy or not fully internalized is something I poke at until I'm satisfied.
It really has helped me develop a sensitivity to what I understand vs what I don't, and the ability to drill into any part of it is amazing.
And yes, it is not that it is just presenting the facts. By me taking control of the direction the questions and answers go, I can flesh out my mental model. I won't retain every little thing it tells me. But I am much farther ahead than before.
But even with Claude, it's it's really the prose getting in the way you can install the caveman plugin or tell it to use that "standard technical English" thing.
Any more detail you can share? Do the others feel more "human"? Are there any that are particularly digestible/human-friendly?
I've been wondering for a while if this is just Claude because I mostly use Claude, so this is very telling.
I tried using a new agent service recently and could tell immediately that it's powered by Claude due to the way it writes.
That's speculative, isn't it
I'm not sure I'm better off with humans though -- I'm not qualified to judge whether a source is a proper authority, not an I qualified to judge whether someone knows enough to point me to a reliable source.
It seems this is a fundamental epistemological problem to which there may never be an answer.
I will say, opus 5 is an egregiously bad case of this, but other LLMs have this too, just less bad.
You can either install that skill or put the Rules section directly in your Global CLAUDE.md for Claude or Personalization setting for Codex and it should cut down the output verbosity by quite a fair bit.
Agreed.
I find Opus 5, and even Fable, to be overly wordy in eg PR descriptions and code comments.
However, I suspect that's more to do with what they are trained to do by default than LLMs in general. I have a little setup where I tell Claude to work together with Codex to tighten up prose and comments, and for me that produces much more palatable text that needs less human editing afterwards.
While my advice is specific to learning about codebases, the way I do it is to have it generate mock data and put it in the local development environment, and give me some exploratory commands, and then ask away. It's a machine after all, so I don't have to read its preceding prose to understand whether it did tell me something, it can just repeat it however many times I ask it, and the hands on commands etc. give me something to actually try and implement.
It's a loop that uses adversarial review to check several dimensions of the writing:
https://github.com/Vibecodelicious/llm-conductor/blob/main/w...
Recently switched to OpenAI and I've gotta say Sol is so much better at writing than Claude. Opus has a distinctive sentence structure and Fable somehow manages to be even more obtuse. The personality of these models really does come through...
This is my biggest gripe with reading AI-generated text as well (ignoring the meta issue of whether it's worth taking the time to read something that an author didn't think was worth the time to write). It's gotten to the point that weird AI-style analogies just take me completely out of the text and kill my interest.
And I can usually tolerate a lot of purple prose.
Sounds like you'd be just as well off link-surfing Wikipedia?
I’ve found the tone of Kimi K3 to be less obnoxious. Unfortunately it doesn’t wholly solve the issue, I don’t think any LLMs out there have a truly pleasant writing style, but at least not every assumption is “load bearing”.
Like I said, I'm essentially continually prompting to refine the material. LLMs certainly continue to append, and never cut back. It just keeps spitting out additional content at me. So that's a bit annoying too. But I can basically get figure out what's going on with a few extra promps.
If youre curious what i've got so far... just be warned it is quite literally AI slop plus me continually prompting for clarification/cleanup etc. : https://github.com/cmoscardi/ai-for-ai
It generates tutorials for you, and serves a webpage that lets you complete them. It does a remarkable job.
It still has a bit of the LLM prose problem, but it does help you fine tune the ‘voice’ it uses.
I've stopped using CC because of it. I find it insufferable.
This problem doesn't get talked about enough and is second only to the hallucination problem IMO.
AI produces so much noise to wade through in order to find signal, and the more expertise you have in a field the more that costs. That noise directly subtracts signifcantly from productivity gains.
And, I think the problem is directly related to the hallucination problem. It feels very much like an effort to kitchen sink the response in order to provide some value among possible hallucinations.
It also seems to be a byproduct of Gen AI operation. It just fundamentally doesn't understand what it's outputting, so doesn't know how to narrow down to the most salient bits.
It's just long. It just doesn't shut up. It's overly verbose. And you can't tell it to be concise or you degrade its quality.
If I ask what an integral is, the correct answer is that it is the continuos analog of a sum, generally used to calculate areas and volumes.
It should really be a single sentence, and then let me ask more about the terms I don't understand, and here's the beauty, in the previous one there can be only 5 terms I cannot know.
An LLM will vomit an entire page or more of explanation which isn't bad per se, but is an answer to something different: "give me a short introductory explanation to integrals". And that's not what I asked.
Try it out, fairly sure that if you out in 100 random words for 30 of them it will just refuse to translate them (it will copy paste the original word into the target language) or it will do silly things like use the target 4th dictionary definition instead of the primary one).
I'm actually going to make a prediction here as well. I think you will soon realize that using LLMs to clarify certain questions or ideas you have will turn out to have frustrations as well. And that you will soon direct those questions to either peers you know in real life or internet forums which are very likely to have a non-AI policy.
Much more likely that people will believe themselves to be an expert in a subject after having had a conversation with Claude about it.
Often "be concise, to the point." is enough, but you can also paste it some stuff you like as an example text and ask to do style transfer.
Literally saying "one sentence response" solves most of this problem.
I'm not sure I follow how this is actually guaranteed? The fact-checking process mentioned just seems to involve asking AI to review its own work.
And even if they are related - if Opus 4.8 always has a 1:100 chance of a specific hallucination - then running the same model twice does indeed dramatically reduce the odds of an error in the final output.
Yes, LLMs can be SOTA for NLP, but you’re going to have to use them to write software or workflows that are more deterministic.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
What triggered my response was the “just review the output with another LLM and it’s perfectly correct”
“"The turtle moves," said Didactylos. "The turtle is a giant reptile that swims through space. It doesn't have to stand on anything. Swimming is what turtles do. The idea that it has to stand on another turtle, and that turtle has to stand on another turtle, is just silly. It's turtles all the way down, and that's a logical absurdity."
Small Gods, 1992
Full story in the book
https://en.wikipedia.org/wiki/Turtles_all_the_way_down
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
I know it sounds silly but 1 layer ends uo being way worse than 2.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
I'm researching causal inference right now, and my main goal was to make sure I understand how to test estimation on synthetic data.
Basically, it's the same way it works with people. If you delegate a task that you don't understand, and you can't have a credibility proof (i.e. doctors, lawyers), then you research a topic well enough to be able to (1) define the task and (2) verify the end result.
In particular:
- I limit it/encourage it to give me single sentence questions
- I sometimes will ask it to tell me a motivating, human-grounded story, when we're starting a new concept: claude responds "Maya is a bond portfolio manager, and her boss has asked her to quickly price in what happened if yields go down. She knows her bond's average duration, a measure in time, but she doesn't have a percentage, which is what her manager wants. How can she give him a percentage number with just a duration figure and the proposed new yield?"
- I'll often ask claude to let me work through it, to derive the thing myself, often resulting in a string of thoughts with "yes/no" trailers, to get the LLM to reply yes or no only, and avoid derailing my train of thought. If yes, my train of thought keeps going. If no, I've got something wrong.
- I'll sometimes stop and have it craft an artifact. I typically say "build me a Brilliant.org-style interactive demo of the topic", especially when we get into the realm of looking at the actual maths of a thing (for which prose and dialog is not optimal by itself AFAICT)
- I'll do this while I'm traveling, while I'm walking, while I'm doing chores.
It's so much fun.
""" In this project, I require a socratically delivered line of conversation. Here's the typical structure to the conversation. I ask some question. You need to factor and reason about how to conduct and deliver a conversation. Best practices would be to limit terminology, or assess with the user whether they have a firm grasp on terminology before you use it. You must be very strict about this, it's unacceptable to just introduce a new concept, actor, phrase or other complication into the conversation without first labeling who what or why it exists for the conversation.
Conversation structure needs to be front-loaded with a brief interview for the user, "you understand X?", "whats your understanding of Y?".
Conversation structure then needs to proceed with single-sentence questions from the agent. User replies with an answer. Sometimes the agent needs to correct the user, but only ever do so with yet another question. """
^^ these are the instructions I have installed at the root of a "project".
Keep in mind, this is claude opus 5 low effort we're talking about, in the "projects" area of the mobile app. Here's the process I use to set up its knowledge:
1. I take screenshots of the textbook on my iPhone, and upload a chapter at a time.
2. I have it summarize the chapter into markdown by analyzing screenshots. You could probably achieve this simpler, if you just had the textbook in PDF.
3. I walk with my boy Clau-crates.
I've done this for a couple of weeks and haven't seen it revert back into its typical context-dumping behavior.
On second read, there's probably some clean up I could do. Thanks for making me pull it out and look at it. Things that could probably be improved:
- tell it to cross-check resources online to further ground itself
- use simple, short sentences (long sentences make the brain blur a bit)
- not be sycophantic (it seems like project mode has discarded with my root-level anti-sycophancy prompt)
I can appreciate using all of the tools at your disposal to learn a new topic, and in no way want to discourage learning. I've used LLMs myself to question my own understandings and it can be helpful.
However, "... 100% accurate and free of hallucinations." isn't a statement someone who just learned the topic is capable of honestly stating.
I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
For example, I’m pretty good at optimizing low level stuff, but right now you can just ask LLMs to do so and they are pretty good at it. They will profile the code and suggest reasonable options like 90% of the time.
Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.
You might think what you're doing is trivial, it may be sessions that flow roughly, "Instrument this, okay this part is slow, profile this part, OK read the profile output and suggest a better approach".
But your experience will be steering it in the right direction, and you're probably unaware of just how much your experience is doing that guiding, as the LLM shoots off at 100mph, you feel like it's taking you with it, but you will be guiding it a lot more than you realise, and that's where learning and experience comes in, even if you're no longer operating at the lowest depth, your knowledge of that layer will be helping.
If nothing else, the experience to know when something is actually slow is a skill in itself. If a function takes 200ms, sometimes that's as quick as it can realistically go, and sometimes that's literally a million times slower than it could be, and there's actual skill and experience wrapped up in knowing what "slow" looks like.
https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23049
So there's this 11 year old issue in a forgotten ffmpeg plugin and I fixed it with deepseek by putting it into a self-testing loop. Probably would have taken me a few weeks to even understand the initial code to begin with. I haven't done C work in a long ass time and have zero knowledge of even what sub pixel sampling means.
With DS4 took me a few days and a couple of dollars. And by few days I mean I checked on it a few minutes every half hour or so a few times.
I don't understand the code it wrote but it's been in production for a while now and no issues so it's good. Ended up speeding up our video processing pipeline by 20-30%.
Sending in the patches but refusing to take responsibility for them is a surefire way to contribute to maintainer burnout. Please don’t do this. Either commit to fixing something and driving the PR to merge, or abstain from it entirely.
The bottleneck isn’t the speed of coding, and what you’re doing here is actively worsening the situation.
But you can see how the parent's comment doesn't really hold, I was able to achieve this while not knowing anything other than what I need fixed and making the LLM test itself towards that goal.
And the craft is loose term, it can mean anything you like to get better at.
“asking the right questions” is also a moving target with each model release
People simply underestimate the value of doing the work and think that the end result is all that matters
https://en.wiktionary.org/wiki/eat_one%27s_seed_corn#English
What is fascinating is how you can witness it at so many levels of organization. One example: Employer executive get enamored with moving from labor to capital. They believe that by using LLMs, they can replace a lot of workers. At my place of employment, we have people that are surprised they can't file a Jira ticket describing a product ask, and have it kick off an implementation. You can build the skill to attempt that, but invariably you'll get back questions like "what do you mean by <x>" and "what do you want to do in this case, a, b, or c?"; questions that a product person or an exec are not well suited to answer.
In the past, programmers did that kind of interpretation and judgment call. So then you're in a quandary; who should do that work? Work that previously, you never imagined was an inherent part of what the replaceable code monkeys do at your beck and call?
And then, how do you hire for that? How do you find the training for the people that are experienced enough with... something... to know what a cohesive error response is, or what kind of telemetry strategy is best for that particular product and organization, what collection of product asks are incredibly complicated for what they're asking and can deliver 95% of the benefits at 5% of the work if we just do this instead, and whether you want to aim more towards thick or thin clients?
Who are those people? Wait, those are programmers? Wait, there's this whole collection of inherently human skills that we devalued, by not appreciating they were always quietly doing that for us in the past?
That's just one example. There's a repeating pattern of discovering where the work truly is, work that was embedded in manual patterns we might not have to involve ourselves with anymore, but is yet still essential. So the nature of our jobs changes massively, but the overall level of employment does not.
At least, not in the medium to long term. There is a lot of painful churn we have to suffer through first.
I'm working on a side-project called tech-professor.com which is a platform for learning. The content is built directly from the source code in your pull request and repositories you follow. So you can quiz yourself and your team based on the code you ship. It's in beta and a lot of changes are still on the way, but if you want to check it out and give feedback, feel free to sign up for free.
I wasn’t even really concerned with optimizing low level code before LLMs and that wasn’t why I was hired either.
However following that low level thread: We can look at the reasonable options and immediately know if they’re reasonable or nonsense. Why? We know the code. Now zoom a level out, where I think our expertise really lies.
Building a complex system isn’t easy. There are customers with requirements, there are budgets, SLAs etc. Sometimes one customer needs X and one needs Y. Our expertise is taking all of this in, and producing something that balances all the different variables. It’s knowing that we’ll expect X events a second so we’ll need Y to ensure we can tolerate failure.
Is it possible LLMs will be able to do all of that too? Maybe. But then why would our customers need the enterprises they pay for?
I have stuff to do now, the value of the knowledge in a year or two isn't important if it solves the issues I have today.
I'm not 'wasting time' but I'm also not really learning.
The more things you understand, the higher the chance you'll spot a situation to use them in the future.
I think the best innovations come from times when someone is uniquely able to combine two of their previous experiences together. The more experiences you have in your back pocket the more combinations you have access to and the more likely you'll have a unique combination when the right problem comes along.
Career-wise, this is terrible. But for me technology was polluted by the ever-growing greed.
For example I’d much rather actually learn assembly, than learn the nitty-gritty details of how LLMs work.
They tell you have "hit the nail on the head" when you really haven't.
They tell you have had a "great insight" when you are really haven't.
They give you the illusion of learning and progress but essentially give you faulty preconceptions will trip you up further down the road.
You can ask the LLM to be more critical and less sycophantic but that only gets you so far:
They want you to continue using, being dependent on and feeding data into the LLM--your independence isn't a priority.
In particular it might be valuable to be in the habit of learning things that one is bad at doing.
Or not.
This is silly. This would be like arguing that encyclopedias made knowing things pointless. I learn new stuff for me.
Professionally, it's important to know enough to know if you're going in the correct direction. Practically, tokens are going to continue to cost money and knowledge can save you tokens.
1. It satisfies you curiosity (and curiosity is always valuable)
2. You can better utilize the LLM to expedite something you now have knowledge about
3. You still improve as an engineer/programmer/prompter/whatever
I still think it's very important not to outsource everything to AI because there is a lot of value in learning and doing things yourself which is an important part of life.
I can tell you that there is an enormous gap in ability between them despite them both using LLMs for daily IR work.
The reasons aren’t complicated. The senior responders have tacit knowledge of how breaches evolve and what to look for which gives them a much better framework for where to employ the LLM.
The juniors will normally start from “here are some logs, look for weird” which is fine but leads to tunnel vision and a lack of confidence in their reporting.
I don’t mandate that anyone do work with or without an LLM. I hire seniors based on experience and juniors based on interest. But my experience has so far been that our best up and comers focusing more on learning the technologies instead of leaving those details to the LLM are developing their intuition and understanding faster and in a more robust manner.
I think that could be called a shortcut, not in learning per se, but in the process of getting to learn.
In my experience it's infinitely easier and faster to learn deep, "boring" things when you understand how they relate to your shallow and wide understanding of all of the related components.
The LLM is merely a tool. And you can use it for domain discovery that enables efficient deep learning at an unprecedented rate or you can develop a cursory understanding of a topic and think yourself an expert.
It’s true of most things. Running, dieting, weightlifting being uncomfortable is a sign of progress.
Learning is uncomfortable. Reading a difficult (for you) text in a language you don't understand is exhausting and confusing. But that is where improvement happens.
Its even worse since Duolingo added a life system (not sure if they still use it), where you were only allowed to make 3 mistakes before having to recharge your energy. If you get everything right, you are not learning, you SHOULD be making mistakes constantly. That shows you're actually being challenged.
One of the teachers on the college I went to had a note on his door with "If you understand everything you're doing - you're not learning anything.
This is the only path to mastery, or understanding if one prefers. There are no shortcuts to a person achieving deep understanding (a.k.a. "Aha!" moments).
Can a tool such as GenAI be beneficial to someone who already has done the work to understand? Absolutely. But it cannot infuse mastery into a person simply by its use.
Only the time and effort a person devotes can do that.
Makes sense.
> I ask it to review the accuracy of the knowledge base it built in the previous step.
Ooookay that sounds good.
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation.
wat.
The main idea is you can do any style you want or like to learn.
Another useful approach has been asking Codex to implement complex things, like a Kademlia DHT or BitTorrent client in a literate style with the explicit purpose to increase understanding by reviewing the source code.
Examples: https://rickcarlino.com/notes/note-dump-and-ai-summaries/ind...
https://github.com/RickCarlino/tiny-bt
For example, “explain how the code in this file works,” I am familiar with the overall codebase, I know the purpose of the file, and I can read it or write tests to verify if I suspect what it’s telling me isn’t correct. Or, if it’s really important, I can overcome my introvertedness and ask the team member who wrote it…but that’s a last resort nowadays, which I am very thankful for. In 99% of cases since at least Claude 4.2 days, Claude and Codex have been very accurate. Gemini on the other hand messes up more frequently and sometimes does weird things like try to delete files it’s not familiar with, at least the 3.6 flash model I’ve been using lately does this. But, code explanations are still good for the most part.
That's actually a fun way to learn processes!
Totally agree, unfortunately careful simulation games are very rare
Otherwise you probably get more confused as you have mentioned.
On the other side, Peter Diamandis describes a situation where a bunch of kids were given a internet-connected computer and they had no teacher. Instead of it there was a “grandma” that checked kids from time to time.
After that there was a knowledge test that revealed “no teacher” approach was more efficient.
But it was a group, not an individual activity…
I can keep asking LLMs to explain a complex topic until I get it. Ask to explain it 10 different ways, explain it using physical analogies, explain it using visualization. If I don't get it, just say that out loud so that they can keep explaining it to me in different ways. We can keep going that until I really get it. That is the value I get the most using LLMs to learn things, especially complex topics.
Or you'll be confused by the slop it generates, which is very conversational but may be just false. Or - you could "get" something wrong and believe it to be the truth because the AI said so.
You can ask the LLM how to do this. Start with a topic you know well to get the mechanism working and trust it well.
I assume this will become less of an issue in the future as there is more trust between the AI tools and me.
Even if you know everything about bicycles their mechanics, components, and how they work you still have to learn how to actually build, repair, or ride one through practice. Knowing, understanding, learning, and practicing are completely different things.
There are also niche areas of expertise that can take years to develop,not just to the point where you know the terminology and jargon, but where you understand the nuances of the field, can recognize the "unknown unknowns", and eventually have the ability to push the boundaries of existing knowledge. Maybe that is what we should really call learning: not simply acquiring information, but developing enough understanding and practical experience to contribute something new to the field.
Ironically, on the same front page of HN, there is a post about Andrew Wiles and this. I don’t think I would be able to comprehend Fermat’s Last Theorem, the Poincaré Conjecture, or Gödel’s Incompleteness Theorems, even with the availability of LLMs.
How does he know?
> Time and time again, when talking to people who rely on ChatGPT, Claude, Perplexity, and other general AI tools, I hear them say, “AI is incredible. It handles nearly everything I throw at them.”
> “What does it fumble with?” I’ll ask.
> “Well, it still gets things wrong when it comes to my line of work.”
https://www.dbreunig.com/2025/04/08/on-ai-observational-comi...
Learning from a book is still the best... Although I have been told that learning a complete subject from a book is now an inproductive use of my time... Perhaps they are right, but I still do it.
The approach that works for me is using Justin's skycak methods he mentions in his books:
https://www.justinmath.com/books/
Check the shorter "Advice on upskilling" or "The Math Academy way" for well researched approach.
So what works for me
- open a project in ChatGPT/Notebook LM - dump all the relevant and highly cited materials (textbooks, papers) - dump the advice on upskilling text or a short summary I've written for the LLM
- create "Learning Goals", that contain what I want to learn, and how to estimate is my level good enough
1) Ask it to create a learning path from the materials, following the approach. Give that to an adversarial LLM for cross check. (just for sanity check)
2) Ask it to create an "entry test" to check what I do know and what I don't
3) Iterate step by step on each module/submodule from the learning path that intermingles the approach of: small theory step + small practical task + small test. Log what's missing/wrong in my dept log. Give the dept log at the end of the session to the LLM to incorporate/create another test/task.
What I have found useful in this approach is that it will generate a lot of practical tests/tasks for me and it will explain a concept in many ways until I understand it. Also it finds some prerequisites I might miss, but based on my tests and debt log unexpected things I thought I understood surface.
So with the limits of LLM and while building a mental map of the relevant parts it's usually enough to spot the hallucinations, but if you apply structured approaches these are minimal. And it's super good, because the number of practice tests and explanations is endless.
The interfaces are a bit clunky, but current multimodal LLMs are ok with images or even hand writing.
I will recommend that structured approach.
Colleagues often suggest podcasts and videos - I very, very rarely listen to them or see them.
The bandwidth is too low. It's not efficient and ultimately I'm bored.
This is a nice project, it looks cute. I watched some of the pages But I want more than that, more information, and faster - still a Wiki fan.
Also, step number 2 in the flow: have the LLM check itself... Naah, I don't believe that.
But you're not the only using gen ai like that. Take care.
Indeed it's often a waste of time to just focus on talking people fully if you want to learn fast, reading and especially deliberate practice are better for that. But if you don't have the time, energy or focus, then listening to interviews in the background can be useful supplementally
The skill then riffs with me, judging my ideas and suggesting alternatives. We go back and forth until something useful comes out of it. This process isn’t unlike how I do normal development.
However, once agreed it breaks the work into “steps”. It then creates a tutorial for me, for those steps, explaining each line, why each change happens etc. I can then ask questions, muse about an alternative idea etc. Then I do the steps, and I’ve learned and gotten what I wanted to get done.
This has been how I’ve been learning Godot and making a game for the past month or so. I didn’t go in blind, I started with a course from GDQuest so I could feel confident guiding the tutorials. I will say though, having a tutor to bounce ideas off of has been really useful.
I still try to figure it out myself, consult the docs, discord etc. But if I’m stumped I’ll run my tutor skill and have some fun.
I have done this for all my work this week and it works quite well.
For one it lets you actually query the LLM as to why, their plans give a high level not every single change and it allows you to correct it as you go and the plan will change.
Of course, you have to be careful with the answers. Especially when the discussions get longer. But usually I see that it is time to stop or to start a new session when the formulae do not make so much sense anymore or when the LLM repeats itself.
But with enough caution, LLMs are really a not-so-bad intellectual sparring partner for discussing ideas and insights.
As for how useful it is to understand thins, I believe it's still useful and hope it will continue to be.
To be clear, I say "inaccurate" rather than "wrong" in this case because even if the information it returns is factually correct to the question being asked, students don't have an understanding of the complexity of the interdependent tectonic, regulatory, and spatial / experiential factors of a building sophisticated enough to ask their questions of the specificity and nuance necessary to get a good output that addresses the entire problem.
Anyway - with the students still learning to ask questions the right way, and the conditionally-incorrect facts making their learning more complicated rather than less, I hit on a strategy for them to use LLM's that seemed to help much better.
I suggested that instead of ask the LLM for the factual answer, or even better for the facts and an explanation, that they ask it to direct them to the proper place in the source material to find the answer themselves. Then, to treat it like a lab partner. IE:
Hey Claude I'm looking for "x."
Claude: "look at foo, bar."
Thank you - chapter (foo) part (bar) table (goo) says "car." However I notice that footnote (hoo) says there's an exception if "dar." Which is what I have. Walk me through this exception...
It seemed to have good results as a guide to understanding the disparate bodies of knowledge that they will eventually have to keep together in their heads and work synthetically and non-linearly through, rather than just as an external source of blindly trusted authority.
Does anyone else use Claude like this?
It's sped up my learning by 10x. I struggled with 'just reading a book.' Take kubernetes. I hemmed and hawed and spent years periodically reading some dry book or blog or official doc, falling asleep, and forgetting while I got busy. Now I'm aggressively working with it, almost like I'm addicted to a gamification, of getting through our learning timeline, and I'm excited to move forward as quickly as possible and pass its tests.
It's like a fake teacher, because I can also ask it to drill into a topic or re-explain itself if it made no sense.
The only thing that worries me is, sometimes I'll say something like, "Um, are you sure about that?", and it'll apologize and correct itself. I barely challenged it!
I'm using LLMs right now to build a terminal browser, a GUI browser, and a PyTorch/LibTorch replacement. It's really fun to be able to learn and make progress this way. It's like reading multiple interactive books, where every concept can be explained again and again until I understand it.
Hallucinations bring to question what you think you've learned. That's going to cost long-term if you labor under mis-apprehensions until you maybe figure out you learned something wrong.
Yes, sometimes its wrong, most times its right, cross checking is fairly easy, not using it because of the possibility its wrong seems a baby with the bath water thing.
I learnt a lot of functional programming from it, stuff I've always wanted to learn, but just didn't have the time and really the sources can be difficult, it really explained things well, and as someone else said in this thread, you can ask questions over and over until you understand, asking a person that (if you can get an expert) would drive them nuts. Maybe my experience isn't typical, its hard to tell, everyone reports something different.
I find LLM's do great for learning when I ask what are the principles, how the main applications work, what are the key drawbacks, where are the growth plates in the field, etc. - the kind of thing a good advisor points to. Sometimes I have to ask it explicitly to use topological order of topics and show relations, which often highlights the gradient changes in the learning curve. For pruning, it's surprisingly good applying philosophical heuristics - Occam's razor, or Derrida's differance (the difference that makes a difference), etc.
And finally, no learning is effective without problem sets, and for those LLM's at times get me over blocking issues.
The degenerate case is memorizing the glib phrases regurgitated back to me; they're helpful and functional enough to get me into real trouble!
LLMs and systems intersection - https://kernelspace.naigap.com
Distributed systems - https://byzantine.play.naigap.com
What is your process for creating these resources?
Also worth mentioning that Matt Pocock has a /teach skill that creates interactive, learning sites for learning a new skill.
https://github.com/mattpocock/skills/tree/main/skills/produc...
OTOH, I am curious if there's a "practical value" to this exercise? If the LLM already contains the information and implementation knowledge to implement the networking stack inside an FPGA by itself, what value do I gain by learning about HDL, TCP, the bespoke Xillinx tooling, reading the documentation, reading papers on the implementation and going through every bit of details and theory. I feel like there's a meta skill that is more worthwhile for "practical value".
One thing I wonder is, do you mentally 'fight back' monotonicity of your interactive tool? All seem to be in 3D space, with low-poly, like in a factory moving through the belt and giving you an information + textual description to read more
But sometimes you want to visualize the charts, or graph of simulations, or maybe even the parts of an item in the rocket.
I had a similar realization a few months back and am working on a tool that generates "mermaid walkthroughs". It is 1000% less pretty but it is fast and is pretty good at explaining how services work or what a code review does or just as a way for your agent to explain some decision to you.
https://github.com/scottrogowski/ariel
I think that would be a way more natural way to explore than being stuck on the classic linear output of a LLM.
> What you get is a beautiful animation that is 100% accurate and free of hallucinations.
How do you make that leap?
> [I ask it to build an interactive thing]
> I then push it to a new repo and enable GitHub Pages for it.
Congratulations. You are an echo chamber for LLMs. Use it to create, and verify, and post to then be scraped and trained on again.
I too, would like to have such a tool for viewing larger projects where the flow can be cumbersome to reason about. One issue I guess would be finding the right level of abstraction in the representation.
I don't think we are even that far when the complexity AI can handle surpasses 99.999% of what humans can handle, where AI make e.g. physics discoveries beyond the grasp of most humans and it will have to "dumb it down" when talking with humans -even physicists- but not with other AIs
Cat also has an awesome podcast with her wife, Ashley Juavinett, Phd, called Change, Technically.[4]
I encourage everyone to check out her work! She’s dedicated her life to helping software developers get the support they need inside organizations to be seen as humans, not just robots.
1: https://www.drcathicks.com#book 2: https://github.com/DrCatHicks/learning-opportunities 3: https://github.com/DrCatHicks/learning-goal 4: https://www.changetechnically.fyi
It's not perfect, but I'm optimistic this will be a useful way to teach/learn in the future.
And, to be clear, I think this will be best utilized within a group/community setting. I don't think it will replace teachers or classrooms.
It reduces friction a ton, but at the end of the day I’m not skipping anything.
But
> What you get is a beautiful animation that is 100% accurate and free of hallucinations
100% free of hallucinations when you're not an expert that can check it is impossible. LLM hallucinations are an unsolved problem.
having things defined/have correct solution to compare for review is useful, and keeps llm on track. Don't think i would trust llm if it were reviewing it all on its own
This is my biggest societal issue with LLMs: they allow you to think that you’ve “learned” a topic because you read a lot of technical terms. I don’t think it bodes well for the future.
0: https://en.wikipedia.org/wiki/Semiconductor_device_fabricati...
I've been working on a similar process of pushing to github pages, but focused more on having "practice sessions" with coding blocks to test content. Using webassembly and mock servers to mock backend endpoints Here's one I built to build a full stack llm chat system in the browser.
https://model-systems-labs.github.io/latent/llm-systems/less...
For background, in 2020 the International Maritime Organization changed fuel standards globally. The intent was to reduce pollution, which is great for human health. But, it turns out some of that pollution was supporting cloud formation and creating a cooling layer globally. And temperatures started to climb when the pollution was removed.
I haven't looked through how it's doing the calculations yet but this kind of visualization is full of possibilities if done right.
Thanks for making this guide!
Here's the code and publications/sources on Github for anyone to poke through: https://github.com/titojankowski/ship-tracks-tycoon
And since the details of ship tracks are a newer topic for me, I'm curious about Gell-Mann Amnesia. Next I'm going to apply this to a topic I understand well and will report back. Stay tuned! (will be tomorrow, bookmark or reply to this thread)
Perhaps basic special relativity could be done this way, or simple derivatives, but definitely not general relativity or integrals, much less PDEs. I guess I was hoping for a 3B1B type output. Oh well.
Last month, I read The Prince and had it make a text adventure campaign for me.
For a lot of other topics, I often just ask it to create a simple python example that I can run.
How do you know if you're learning this for the first time? Very risky to learn from LLMs. I've done it, but you have to keep your wits about you. Lots of "oh of course you're right - what I just told you was completely wrong".
God does everything have to be productized and glorified as if you’ve invented a new way of learning. Read some books!
Inthink this was always the best way to learn. But it used to require immense work for a teacher.
My compiler course is a great example - program in plug in a stage of a compiler.
These exercises can be made on demand and incredibly easy now.
You're just memorizing a nonsensical recipe. What are the constraints? Why do we do X rather than Y? How does a particular thing scale? etc.
All you're doing is fooling yourself into thinking that you've acquired some knowledge. When in reality you haven't even learned the basic mental model to reason about this stuff.
You've learned something when you have a mental model that makes correct predictions. Until then you've memorized it at best, and as with most memorized things it will decay exponentially and will be gone from your memory soon enough.
Do you know it is free of hallucinations because you crossed checked it with the source material or because you told the LLM "don't hallucinate"
Surprisingly effective.
(Yes I confirmed it was ok to use AI with the material)
I also struggle with LLMs explaining things, but for the opposite reason.
I consistently have problems to get short, precise but plain/simple answers.
Instead I'm overwhelmed with walls of texts, often filled with jargon that is a mixture of imprecise and unneeded.
The style at which I learn better is by asking about stuff interactively. I ask you what something is, you give me a 3-4 sentences top answer. Then I explore and dig into the topic from your answer on the things I want to know better.
Then I went back to a book written by humans (Eric Nikitin's "Realm of Oberon").
I am pretty sure this person is
a. Ignorant of the fact, that AI is designed to give just answers, they do not care whether they’re true or not
b. Have never Cross-checked Information given by AI with reliable sources