WhichBox – AI Assistant App

Do you remember which one of those boxes in the garage has your old phone? Or that toy your child wants to play with again? Or the tripod you need for your weekend trip? Oh what about that massager that you could really use right now! You would probably have to dig through those boxes to find it. Same problem when moving. There are always some boxes that you don’t want to open right away, but you wish you could open just the right one when you need something.

I have created WhichBox – your AI Assistant to find that box. It uses latest Vision AI models to help you find things quickly. Here’s what you do –

  1. Take pictures of the content inside each labelled box to create an inventory.
  2. Use WhichBox to easily identify the box containing the item you are looking for.

You can check it out here – https://whichbox.streamlit.app/

The demo has four photos of labelled boxes with some content in them. Note that there can be multiple photos of the same box. You can take photos as you are filling up the box to capture things at the bottom.

You can ask for a specific thing, like “Find Fitbit” or just “fitbit”

You can go for a general category, like “Camera Equipment”

You can get all boxes containing “USB Adapters”

You can look for “Groot” or if you can’t remember the name of a specific toy then you can look for “all toys of movie characters”

For now, you can bring your own API Key and use your own photos to try this out.

WhichBox : https://whichbox.streamlit.app/

AI Powered Bookshelf

Bookshelf is a Generative AI application built as a rudimentary, but fairly capable, RAG implementation written in python. It can use an open source LLM model (running locally or in the cloud) or a GPT model via OpenAI’s API.

  • The application is created using streamlit.
  • I used llama-index for orchestrating the loading of documents into the vector database. Only TokenTextSplitter is currently used. It does not optimize for PDF, html and other formats.
  • ChromaDb is the vector database to store the embedding vectors and metadata of the document nodes.
  • You can use any open source embeddings model from HuggingFace.
  • Bookshelf will automatically use the GPU when creating local embeddings, if the GPU is available on your machine.
  • You can use OpenAI embeddings as well. There is no way to use a specific OpenAI embedding model or configure the parameters yet.
  • Use OpenAI API or any OpenAI compatible LLM API (using LMStudio, Ollama or text-generation-webui) of your choice.
  • There is a live demo on streamlit cloud – https://bookshelf.streamlit.app/
  • The demo allows only OpenAI integration. You can run it locally for accessing Open Source embedding models and LLMs.

Live demo – https://bookshelf.streamlit.app/

You will need your OpenAI api key for the demo.

If you are running it locally, you will have the option of using an Open Source LLM instance via an API Url. In the screenshot, I am using an open source Embedding Model from HuggingFace (sentence-transformers/all-mpnet-base-v2) and The local LLM server at http://localhost:1234/v1

Collections tab shows all collections in the database. It also shows the names of all the files in the selected collection. You can inspect individual chunks for the metadata and text of each chunk. You can delete all contents of the collection (there is no warning).

You can modify the collection name to create a new collection. Multiple files can be uploaded at the same time. You can specify if you want to extract metadata from the file contents. Enabling this option can add significant cost because it employs Extractors which use LLM to generate title, summaries, keywords and questions for each document.

On the Retrieve tab, you can query chunks which are semantically related to your query.

On the Prompt tab, you can prompt your LLM. The context as well as the Prompt Template is editable.

Here is an example of using the context retrieved from chunks in the Vector database to query the LLM.

This inference was performed using Phi3 model running locally on LMStudio.

Code is on Github – https://github.com/ashtewari/bookshelf

Have fun!

Prompt Engineering and Security – Custom GPT

I have been delving into Advanced Prompt Engineering and Security techniques for Large Language Models (LLMs). As an exercise, I have created a custom GPT in ChatGPT to help practice spelling of English words. Spell It GPT is secured with Advanced Prompt Engineering techniques to guard against common attack vectors, including Direct Prompt Injection, Prompt Leaking, Role Playing, Simulation, DAN and Code Injection.

Spell-It-GPT

Play with the Spell It GPT and try to break it. It is not impossible but (probably) fairly difficult to do 😉 Regardless, practice spelling and have fun!

Switch to Voice Mode in ChatGPT Mobile App to practice spelling and put headsets on for best results!

Spell It: https://bit.ly/3Ua0oib

Here are some resources you can learn more about Prompt Engineering and Security:
Advanced Prompt Engineering Techniques
OWASP Top 10 for Large Language Model Applications
Educational Resources
Adversarial Prompting in LLMs

KHEL Coin on Ethereum

KhelCoin (KHEL) is an ERC-20 token available on Ethereum blockchain. It is a Smart Contract created with Solidity, OpenZeppelin, Truffle, Ganache, Remix, web3.js and VSCode. It is currently available on Rinkeby test network. It will be launched on the Mainnet in the near future.

You can get KHEL coins from this website – KHEL Coin ICO

%KhelCoin 
The Game Coin 
Metamask is needed to view your balance or buy KHEL Coins 
Click here to download Metamask 
METAMASK

Head over to the ICO website. You will be prompted to install Metamask and connect to your account on Rinkeby testnet. Click on “Buy KHEL Coins” button to get your own coins!

Source code is on github – https://github.com/ashtewari/khelcoin

Puzzle Icon Credit: https://www.flaticon.com/free-icons/puzzle

Universal Windows Platform Application on Raspberry Pi

Universal Windows Platform (UWP) provides a common app platform on every device that runs Windows 10. The core APIs in UWP are the same on all Windows devices – including Desktop PC, Mobile Phone, XBox, Hololens, IOT devices and others

You can target specific device capabilities of a device family with extension SDKs, but you don’t have to do that if you are only using core APIs. Those core APIs include a very impressive set of UI capabilities. What that means is that you can create a UWP application using C# and XAML which will run on an ARM based processor on Raspberry Pi – because Windows 10 IOT Core runs on that.

Universal Windows Platform (UWP) has come a long way. I decided to dust off my good old Raspberry Pi 2B and take it for a spin last weekend.

The goal was simply to create a Hello World UWP application, deploy it on Windows 10 IOT running on Raspberry Pi (RPi), without using any device specific extension SDKs. Here is the stack – 

Create a simple UWP application in Visual Studio 2019. 

Install Windows 10 IOT Core on Raspberry Pi 2B and connect the device to a 50″ screen TV. 

Select ARM in the target platform drop down and Click on Remote Machine to Run/Debug Application on RPi device.

Select RPi device for deployment.

Deployment of UWP on RPi is complete. 

UWP application running on RPi. 

I updated the UWP application to include Syncfusion Chart control. Did you know that you can get a Community License for the entire product line (including Blazor, Xamarin and UWP controls) from Syncfusion?

UWP application is now running on RPi, attached to the Visual Studio debugger running on the development machine (Windows Surface). 

No device specific extension SDKs were referenced, just the core UWP APIs were used.

Code is on github.

PackRaft with Xamarin and Microsoft Azure

PackRaft is a cross-platform mobile application built with Xamarin. It integrates with Microsoft Azure services, such as Azure Table Storage and Azure App Service. It is created with Xamarin.Forms for iOS and Android (Windows version will be coming soon).

packraft-android-riders   packraft-iphone-assignments

The purpose of creating this sample application was to expand expertise in cross-platform mobile development with Windows Azure integration, specifically with Xamarin.Forms and Azure App Service. With Xamarin.Forms, it is possible to achieve close to 100% code reuse across these platforms for certain type of apps. Azure App Service provide a set of rich cloud backend services for your mobile applications.

PackRaft-Platform

PackRaft allows the user to setup a fleet of vehicles of various seating capacities. Groups of riders are assigned to these rides according to the count of riders in each group. You can specify that certain groups be kept together on a single vehicle – and the app tries to honor those requests if possible. I did this via a rudimentary implementation of a bin packing algorithm. This implementation is deployed as an API App in Microsoft Azure.

PackRaft uses Azure Mobile Services SDK to persist Fleet data to Azure Table Storage and to authenticate users with Facebook and Microsoft accounts. Twitter and Google account authentication is also supported by Azure Mobile Services SDK out of the box and those will also be enabled in PackRaft soon.

PackRaft is available on Android, iPhone and iPad.
Get it on Google Play

Download iOS app on App Store