Disclaimer: This content reflects my personal opinions, not those of any organizations I am or have been affiliated with. Code samples are provided for illustration purposes only, use with caution and test thoroughly before deployment.
Llm-chain is a Rust crate that help you create advanced LLM applications such as chatbots, agents, and more. It supports various drivers, such as OpenAI, llama.cpp, and llm, that can connect to different APIs or run models locally. llm-chain allows you to easily switch between different drivers and options without a complete rewrite of your code.
One of the challenges of using llm-chain is the cost associated with invoking the LLMs. Depending on the driver you use, you may incur either an API fee or a compute resource cost for running your own model. The local models usually requires pretty powerful machines, and the setup process might be a little complicated.
Hi, everyone! I hope you are all doing well and enjoying Rust as much as I do. I have some exciting news to share with you today: I have just published my new book, Practical Rust Projects, 2nd Edition.
I would like to acknowledge the contribution of my co-author, Andrew Rzeznik, who took over the project after I had to stop for personal circumstances. He did a fantastic job in finishing the book and making it even better than I could have imagined. I am very happy with the final outcome of our collaboration, and I hope you will find our book useful and enjoyable. Thank you, Andrew Rzeznik, for your hard work and dedication!
Recently I’ve been contributing to llm-chain, a Rust library for working with large language models (LLMs). It’s similar to Python’s LangChain. I contributed this tutorial to the official website for setting up a simple llm-chain using LLaMA models running locally using the llama.cpp project. I’m also publishing the same tutorial in this blog for archiving.
Tutorial: Getting Started using the LLAMA driver
In this tutorial, you will learn how to set up an llm-project using the LLAMA drive. If you wish to use the other drivers you can skip this part of the tutorial.
I like Brave’s SpeedReader feature. It is a new approach to reader modes that removes unnecessary elements from pages it recognizes as articles and displays them in a simplified format as part of the rendering pipeline. Because it’s done during rendering, not after the page have rendered (as most reader mode extensions does), it delivers performance gain and reduce the network bandwidth required.
However, there is one thing that bothers me about SpeedReader. It follows the browser theme, so if you choose dark theme the SpeedReader theme is always white text on black, which is hard to read for me. I prefer a sepia theme for reading articles, as it is more soothing for my eyes. Unfortunately, there is no UI for changing the SpeedReader theme. However, I found this github issue. It shows that the theme has actually been implemented, but there is no way to access it from the browser settings. If you set the data-theme attribute on document.documentElement using JavaScript, you can toggle between dark, light and sepia themes.
In this blog post, I will show you how to use AWS CLI and SageMaker JumpStart to create your own private large language model (LLM) on Amazon SageMaker. You will learn how to deploy a pretrained model from the Hugging Face Transformers library, and how to use it to generate text with custom instructions using AWS CLI.
