by: vintermann

Maybe impressive in one way, but I&#x27;m also pretty sure a simple n-gram Markov model (a la Niall on the Amiga) would have a lower loss on the test set.<p>Transformers don&#x27;t scale <i>down</i> very well, in my experience - I used to train local models all the time as new ones were released, as I recall transformers were the first ones I couldn&#x27;t get better results out of with my limited training data and GPU.

4/21/2026, 10:43:29 AM

by: wk_end

&gt; 25K parameters is about 70 million times smaller than GPT-4. It will produce broken sentences. That&#x27;s the point - the architecture works at this scale.<p>Since it seems to just produce broken and nonsensical sentences (at least based on the one example given) I&#x27;m not sure if it <i>does</i> work at this scale.<p>Anyway, as written this passage doesn&#x27;t really make a whole lot of sense (the <i>point</i> is that it produces broken sentences?), and given that it was almost certainly written by an AI, it demonstrates that the architecture doesn&#x27;t work especially well at <i>any</i> scale (I kid, I kid).

4/20/2026, 10:23:06 PM

by: daemonologist

You can chat with the model on the project page: <a href="https:&#x2F;&#x2F;indiepixel.de&#x2F;meful&#x2F;index.html" rel="nofollow">https:&#x2F;&#x2F;indiepixel.de&#x2F;meful&#x2F;index.html</a><p>It (v3) mostly only says hello and bye, but I guess for 25k parameters you can&#x27;t complain. (I think the rather exuberant copy is probably the product of Claude et al.)

4/20/2026, 11:40:01 PM

by: mixmastamyk

Just reminded me of the random sentence generator program on my Vic-20. I had changed most of the words to all the bad words a preteen could think up. So many laughs with the neighborhood kids.

4/20/2026, 11:52:09 PM

by: arketyp

I love these counterfactual creations on old hardware. It highlights the magical freedom of creativity of software.

4/21/2026, 6:58:03 AM

by: borsch_not_soup

Interesting, I’ve always thought neural network progress was primarily bottlenecked by compute.<p>If it turns out that LLM-like models can produce genuinely useful outputs on something as constrained as a Commodore 64—or even more convincingly, if someone manages to train a capable model within the limits of hardware from that era—it would suggest we may have left a lot of progress on the table. Not just in terms of efficiency, but in how we framed the problem space for decades.

4/21/2026, 1:36:37 AM

by: classichasclass

If you&#x27;re running this in VICE, run it under the SuperCPU with warp mode on.

4/20/2026, 11:02:02 PM

by: anyfoo

This would have blown me away back in the late 80s&#x2F;early 90s.<p>(Or maybe not, if it doesn&#x27;t perform better than random, I haven&#x27;t actually tried it out yet. Some more examples would have been nice!)<p>I wonder how far you could push this while still staying period correct, e.g. by adding a REU (RAM Expansion Unit), or even a GeoRAM (basically a REU on steroids).<p>SuperCPU would also be an option, but for me it&#x27;s always blurring the line of &quot;what is a C64&quot; a bit too much, and it likely just makes it faster anyway.

4/20/2026, 11:17:40 PM

by: rahen

A little disappointed to see PyTorch + Claude here. I was hoping for some &quot;demo-scene&quot; hand-crafted 6502 assembly, and hopefully training on the C64.

4/21/2026, 7:44:30 AM

by: djmips

Dissapointed - there was no 6502 code in the GitHub repo.

4/21/2026, 3:25:45 AM

by:

4/21/2026, 12:38:04 AM

by: brcmthrowaway

How does this compare to ELIZA?

4/20/2026, 11:23:59 PM

by: harel

Eliza called, and asked if we saw her grand kids...

4/20/2026, 10:18:05 PM

by: Vaslo

Load”*”,8,1<p>Brings back memories

4/21/2026, 1:32:56 AM

by: Lerc

Ok now we need 1541 flash attention.<p>I&#x27;m not sure what the venn diagram of knowledge to understand what that sentence is suggesting looks like, it&#x27;s probably more crowded in the intersection than one might think.

4/20/2026, 10:54:05 PM

by: bighead1

i hate ai, and i love the c64, but i&#x27;ll allow it.

4/20/2026, 10:20:04 PM

by: ghstinda

but can you make mac keyboards feel like a c64c?

4/20/2026, 11:04:20 PM