Tag: github

AI Powered Bookshelf

Bookshelf is a Generative AI application built as a rudimentary, but fairly capable, RAG implementation written in python. It can use an open source LLM model (running locally or in the cloud) or a GPT model via OpenAI’s API.

  • The application is created using streamlit.
  • I used llama-index for orchestrating the loading of documents into the vector database. Only TokenTextSplitter is currently used. It does not optimize for PDF, html and other formats.
  • ChromaDb is the vector database to store the embedding vectors and metadata of the document nodes.
  • You can use any open source embeddings model from HuggingFace.
  • Bookshelf will automatically use the GPU when creating local embeddings, if the GPU is available on your machine.
  • You can use OpenAI embeddings as well. There is no way to use a specific OpenAI embedding model or configure the parameters yet.
  • Use OpenAI API or any OpenAI compatible LLM API (using LMStudio, Ollama or text-generation-webui) of your choice.
  • There is a live demo on streamlit cloud – https://bookshelf.streamlit.app/
  • The demo allows only OpenAI integration. You can run it locally for accessing Open Source embedding models and LLMs.

Live demo – https://bookshelf.streamlit.app/

You will need your OpenAI api key for the demo.

If you are running it locally, you will have the option of using an Open Source LLM instance via an API Url. In the screenshot, I am using an open source Embedding Model from HuggingFace (sentence-transformers/all-mpnet-base-v2) and The local LLM server at http://localhost:1234/v1

Collections tab shows all collections in the database. It also shows the names of all the files in the selected collection. You can inspect individual chunks for the metadata and text of each chunk. You can delete all contents of the collection (there is no warning).

You can modify the collection name to create a new collection. Multiple files can be uploaded at the same time. You can specify if you want to extract metadata from the file contents. Enabling this option can add significant cost because it employs Extractors which use LLM to generate title, summaries, keywords and questions for each document.

On the Retrieve tab, you can query chunks which are semantically related to your query.

On the Prompt tab, you can prompt your LLM. The context as well as the Prompt Template is editable.

Here is an example of using the context retrieved from chunks in the Vector database to query the LLM.

This inference was performed using Phi3 model running locally on LMStudio.

Code is on Github – https://github.com/ashtewari/bookshelf

Have fun!

KeyNode with Node.js and Microsoft Azure

KeyNode is a application to issue and verify software license keys. Technology stack for KeyNode is Node.js, MongoDB and Microsoft Azure.

I had built this functionality with C9.io (a cloud-based IDE with a built-in source code repository and debugger), mongohq (MongoDB as a service – now part of compose.io) and appfog (Cloud PAAS built on top of CloudFoundry). It used SMTP/gmail to email license files. That was the version I created a couple of years ago to issue tamper-proof signed xml license files for CodeDemo (a code snippet tool for developers, presenters and instructors).

For KeyNode (open source) I switched to a different toolset : Visual Studio Code and Windows Azure, simplified the code to remove signed xml file and open-sourced it on GitHub. Signed xml allowed offline verification in CodeDemo (a Wpf/Desktop app). Removing signed xml requires verification to happen online. I am working on adding the web endpoint for verification of license keys. This version uses SendGrid to email license keys. KeyNode is deployed as a Windows Azure Web App. The Azure Web App is on Continuous Deployment feed from the source code repository on GitHub.

I created and tested this Node.js application locally without IIS and deployed it as an Azure Web App without making any changes to the code at all. Node.js applications are hosted in Azure under IIS with iisnode. Iisnode is a native IIS module that allows hosting of node.js applications in IIS on Windows. Read more about iisnode here. Iisnode architecture also makes it significantly easier to take advantages of scalability afforded by Azure.

KeyNode is a work in progress. My plan is to use this as the basis for further explorations in the following areas :

  • DevOps, Docker and Microservices (at miniature scale of course!)
  • Create a Web UI with Express (a Node.js web application framework)
  • Integrate with Azure Storage/Queues
  • and more…

I invite you to check out the live site on Azure and fork it for your own experiments : KeyNode on GitHub.

Resources :

Photo Credit : Piano Keyboard (www.kpmalinowski.pl)

The Site44 Workflow

A light weight development workflow with real-time website deployment.

I recently built a sample website to illustrate how clean, semantic html markup can be maintained when using Bootstrap’s grid system. The solution is to use a css pre-processor to incorporate Bootstrap’s LESS based mixins into your own .less files and push the Bootstrap instructions down into your stylesheets. There are two ways to “compile” .less stylesheets – use a stand-alone LESS compiler or use less.js. I found it very convenient to use less.js (note that it is not recommended in production deployment). As I started working on developing the sample code I found it a bit cumbersome to work with an entire web application project in Visual Studio, considering I was working with some really simple sample client-side html, css. As I craved for an alternative, I stumbled on to a development workflow that is incredibly simple and a lot of fun. I call it the Site44 workflow. Site44 turns your dropbox folders into websites. And it is awesome! Here is what you do –

1. Sign into Site44.com using your dropbox credentials.

2. Create a new site (all you have to do is come up with a name). I named it “ash”. A sub-folder with this name will show up in your dropbox folder.

3. Drag this folder to your Github for Windows screen and drop it there to create a github repo in that folder and push it to github.

4. Smile and write code.

As you save your code. The changes are deployed in real-time to your website. You commit to your github repo as you please. If you revert to a different version/branch of our code from your git repo, that version will be deployed (almost) instantly to your website. I wish there was a .site44ignore feature in Site44, just like .gitignore. That will allow me to keep my .git folder (and some other files) from getting published to the website. Other than that, this worked out really well for me.

I wrote about the experience of extending Bootstrap with LESS here : Bootstrap with LESS.

Hat tip to Justin Saraceno for introducing me to site44.

 

Protecting Your Api Keys

I am working on a Windows 8 app (details to follow in a subsequent post) and the code is published in a public repo on github. My app uses third-party APIs and after I committed the first cut to github, I realized that I had included my api keys in the code. The whole world had access to my keys. I did not want to publish the developer keys for those APIs to the entire world.

When the app will be released and distributed, those keys will need to be included in the app somehow. Once the keys are out there they can not be 100% protected from a determined mind. So, why bother? Why would I want to hide the api keys in the source code? Here are some good reasons –

1. It might be illegal to put the keys out there in plain sight for the whole world to see.
2. Developer keys may be throttled or have other restrictions on how many times they can be used per day or per minute.
3. The keys might allow access to expensive cloud computing resources.
4. The keys might allow access to confidential/sensitive customer data.

First, I had to take my keys back from git repo. Can you really remove information from a public git repository? Yes, you can, using git filter-branch. Here is how – https://help.github.com/articles/remove-sensitive-data. It worked! I successfully rewrote the history! My past commits don’t have those file(s) anymore that had my private api keys.

Next, I made sure that I don’t make this mistake again –

1. I added a new file ApiKeys.cs to the project.
2. Exposed the api keys as constants from a static class in this new file.
3 Added ApiKeys.cs in .gitignore file, to prevent this file from being committed to the repository.
4. Added instructions in ReadMe.txt for external developers to include their own keys.

This is not an ideal solution. If you are using a continuous build server, this technique will obviously not work. The code will not compile as-is, a file must be added to the project before it will start compiling. This works for me for now, but I am still looking for a better solution.

Solution to the fetch puzzle

Here is a brute force solution to the fetch problem –
Basically, at each step there are three possibilities :
1. You can fill a bucket.
2. You can transfer water from one bucket to the other one.
3. You can dump out the water from a bucket.
In this brute force solution, I try each one of these steps and then try all three again after each one of the previous steps. And on
and on untill I get the required amount of water in one of the buckets.
Check it out. Source code is on my github repo –
https://github.com/ashtewari/fetch

Here is a brute force solution to the fetch puzzle.

The puzzle goes like this – You have two buckets. A 3 gallon bucket and a 5 gallon bucket. Buckets are not marked or graduated. You are to fetch 4 gallons of water in a single trip to the river. How will you do it?

Basically, at each step there are three possibilities :

  1. You can fill a bucket.
  2. You can transfer water from one bucket to the other one.
  3. You can dump out the water from a bucket.

In this brute force solution, I try each one of these steps and then try all three again after each one of the previous steps. And on and on until I get the required amount of water in one of the buckets.

Check it out. Source code is on my github repo – https://github.com/ashtewari/fetch