data-processing pipeline

Flowchart of the process

For a client at our AI-agency, I built an automatic data processing pipeline for analyzing and processing unstructured webpage into their Content Managment System.

Articles from the web were scraped using Selenium and BS4.

The data was then processed using OpenAI and Huggingface models.

Integration into Airtable allowed the results to be easily interpreted by humans.

Everything was powered by event-driven architecture on AWS using Queues, Webhooks, etc.

ChatRPG

My web-dev journey started with a text-adventure generation engine built over the OpenAI API.

Using some clever prompting I could provide ambient music/visuals fitting to the current situation, as well as a inventory and health system.

It was built using Flask, served by Gunicorn + Nginx, and deployed on an EC2 instance.

This was also the first time taking care of Domain, DNS, and SSL certificates.

I tried to market this thing with some shitty Reddit posts, which obviously didnt work.

development boilerplate

Screenshot of Graphic demonstating pros of the boilerplate

Thinking I was gonna get get rich like Marc Lou (some guy from Twitter making thousands), I built ShipPil, a Boilerplate for Flask developers.

The infrastructure (runtime on Lambda Functions and DynamoDB as database) is defined as code using the Serverless Framework.

It included a full bundling setup with ESLint and PostCSS, a CDN, auth, and payments.

A strong emphasis is also put on security, taking care of things like CSRF, CSP, and input cleaning.

The documentation is actually pretty good as a ressource for learning the concepts behind it - to be trully useful it was designed like a course so the user could better work with and modify the boilerplate.

more agency stuff

For another client I built an in-house ChatGPT-like client with a RAG system. The Vector Store and the Model itself (LLama3.1 70b) was deployed locally.

After successfully this as service for multiple clients, we are also currently developing a platform for alt-tag generation. You simply upload a CSV export from your CRM and get thousands of SEO and accessibility optimized tags.

☘