So after I saw a video of Llama3 8b doing 800-1000 tokens/s I instantly thought it would be phenominal to have that on my laptop instantly giving me responses that I could select or discard with the added side benefit would be to learn some new tech.

I have once coded a small AI trained on like 100.000 items trying to predict housing prices, let me say it was not great.

But again I don't have billions upon billions of records to train on nor the compute to train a decent model but I like the control over my coding environment.

The first goal

Having llama 8b running locally autocompleting code snippets.

Yeah I know the barrier are low but is most def a useful starting point. I personally dont believe in setting unobtainable goals because failing at something hard is not as great as succeeding at small increments.

Lastly I intent to make a guide on my complete journey sub and like to get notified when new content is published

Banner video is from groq.com a great service where you can test these open source models like llama not sponsored by them just wanted to credit