
<Image
  alt={`Dominate.codes blog post hero`}
  src={`/static/images/dominate-codes/hero.png`}
  width={1920}
  height={1080}
  priority
/>

[Dominate.codes](https://dominate.codes) is an online multiplayer game that challenges you to solve computer science questions both against time and other players in real time. After choosing to sign as a guest or using a third party account, users can elect to create a room or join an existing room using a short code. Questions pop up on the map in random places with different shapes and answering questions correctly colours the map in your colour. The first player to dominate the map wins!

## What was this project for?

This project was built for the Level-Up Society Hack, a hackathon organised by ShowCode centered around levelling up society using gamification. While we originally submitted to the learning category, we ended up winning the overall grand prize in this competition!

### How was the competition judged?

The competition was judged on the uniqueness, viability, gamification aspect, execution and pitch submitted with each project and there were participants from around the world!

## Moving quizzes from boring lists to something more

We've all spent hours doing quizzes throughout our academic lives from an early age. But they were always boring lists.

The concept behind [dominate.codes](https://dominate.codes) is to make the questions more engaging and interactive using information visualization and gamification to present the questions in a conquer-the map format.

### Map and game block design

As the picture shows, we show a low poly, block based map of an arbitrary shape and use custom shapes inside the circles to represent the amount of the map that completion would conquer. Reminiscient of the classic atari games where speed and accuracy determine the winner, dominate.codes puts users against the test of time and with multiple players, let's them compete to win.

<Image
  alt={`Question Shape Logic`}
  src={`/static/images/dominate-codes/question-shape-logic.png`}
  width={1920}
  height={850}
  priority
/>

### Custom map generation

Although we started with relatively simple maps with different shapes like in the screenshot above, we then extended the functionality of the site to enable us to generate custom maps using any black and white images. If you check out the site, you should see everything from Pterodactyl's to world maps!

<Image
  alt={`Automated map grid generation`}
  src={`/static/images/dominate-codes/map-generation.png`}
  width={1920}
  height={550}
  priority
/>

## How was it built?

<Image
  alt={`System Arhictecture Diagram for Dominate.codes`}
  src={`/static/images/dominate-codes/system-architecture.png`}
  width={1920}
  height={1080}
  priority
/>

The application was built over two full days of intense, non-stop hacking with: ReactJS, Gatsby, Firebase, TailwindCSS, ExpressJS, NodeJS, ReactJS, and Socket.io.

- ReactJS as the de-facto JavaScript library for building user interfaces
- NodeJS for the server side code and custom map generation
- Socket.io to enable real time communication, multi-player games and enhanced user experience
- Firebase to enable user accounts, SSO, and as a database.
- GatsbyJS as our blazing fast React framework for performance, scalability, security and accessibility
- TailwindCSS to leverage the benefits and speed of the utility first CSS framework

During this project, on Saturday we worked on the question logic and question bank, map interactions, the game loop, ux improvements and branding/design. On Sunday, the hacking didn't slow down one bit and we shifted over to working on the sound design, leaderboard page and integration along with carrying out user testing for a ux overhaul of the game experience.

## User Testing - even in a short Hack

Whenever I'm working on a project - whether its full time, a side project or a hack - I always make sure to remember what we're building for. Engineers often forget that they're building products for others to use so even in this short two day hack, we made sure to carry out a few user testing sessions to optimize the game playing experience and sanity check the product we were building.

<Image
  alt={`User testing`}
  src={`/static/images/dominate-codes/user-testing.png`}
  width={1920}
  height={1080}
  priority
/>

## What did I learn?

This was my first time working with Web Sockets and building a more complex state with the React Context API - I've since used it for far more interesting things but at the start it took a while to get my head around. Working with Web Sockets was a great experience and I've loved working with them since because they're such a powerful tool to have in your arsenal!

It goes without saying but hackathons are such a great way to stay in touch with learning. Hackathons largely made up my university experience and this was my first hackathon since I finished university and from the moment I finished, I couldn't wait for the next one.

<Image
  alt={`High level application preview inside a computer screen`}
  src={`/static/images/dominate-codes/high-level-preview.png`}
  width={1920}
  height={1080}
  priority
/>

If you haven't already jumped off to try it out and read to the end of this post, please check out [Dominate.codes](https://dominate.codes) and let me know what you think on twitter [@yannispanagis](https://twitter.com/yannispanagis)!


Online Collaborative multi-player quiz game with interactive maps

Dominate.codes


<Image
  alt={`InterviewToast blog post hero image`}
  src={`/static/images/interview-toast/hero.png`}
  width={1200}
  height={725}
  priority
/>

# InterviewToast

InterviewToast is an interview preparation, feedback and assessment tool that’s designed to help you make sure that at your next interview, you won't be toast!

INSERT GIF WITH BEING TOAST

## How does it work? 🤔

This project was built with a friend for the DeveloperWeek 2022 Hackathon in a week and fulfils a couple of functions:

- _Interview Preparation Tool_: There are a lot of great tools out there to help you prepare for the coding part of the application process. However, a lot of the interviews in the process to get any new job aren't just about code and there aren't a lot of tools out there specifically designed to help you with this part of the process. By leveraging Symbl's conversation intelligence AI in our video application, InterviewToast goes beyond simple speech-to-text and provides real time insights and extracts topics of discussion, sentiment and conversation analytics such as talk-to-listen ratios.
- _Interview Feedback Tool_: By sourcing questions either directly from the candidate which they want to practice, InterviewToast computes performance ratings and provides quantitative and qualitative feedback on practice interviews for candidates.
- _Interview Assessment Tool_: InterviewToast uses Agora to provide real-time transcriptions, key-word and profanity detection, and analysis on live interviews. Companies are under increasing pressure from regulators to provide objective feedback after being sued for not hiring candidates. InterviewToast offers a third perspective that leverages real-time data analytics and impartial objective feedback to improve the experience for candidates and interviewers alike. On top of this, the platform aggregates data on the questions that interviewers ask in their interviews to eventually funnel these back into the question database. This way we can make sure that candidates are practicing with the most relevant, up to date, and likely to occur questions.

This project was exciting for a few reasons - it was my first opportunity to experiment with WebRTC, which is am open source project that is used by a lot of applications, in particular video-chat applications, for real time communication.

## 🚀 How we built it

The application was built with:

- Symbl.ai’s for sentiment analysis, topic extraction, conversation analytics, entity and intent recognition, and real-time insights APIs.
- Agora for video calling
- ReactJS as the de-facto JavaScript library for building user interfaces
- GatsbyJS as our blazing fast React framework for performance, scalability, security and accessibility
- Gatsby Cloud Functions to bring an entire backend to InterviewToast - without managing a backend
- Firebase to enable user accounts, SSO, and database.
- TailwindCSS to leverage the benefits and speed of the utility first CSS framework .
- TensorFlowJS Pose-Detection for slouch detection, see in detail below

These pieces all came together nicely in the simple architecture diagram below, which was perfect for a short, time-limited hackathon outside of work hours:

<Image
  alt={`InterviewToast architecture diagram`}
  src={`/static/images/interview-toast/system-architecture.png`}
  width={1200}
  height={725}
/>

## Bonus: Pose-Detection with TensorFlowJS

Did you know that TensorFlow has a JavaScript API? Well it does and its very fun to experiment with…Conversational insights go beyond just the audio in conversations. In fact, only 7% of the interpretation of a conversation is verbal, 38% is vocal and a whopping 55% is /_visual_/. To make use of this component of the practice sessions, we used the TensorflowJS Pose Detection package’s state-of-the-art models for real-time pose-detection.

The MoveNet model detected up to 17 keypoints on the body and could run at 50+ fps on modern phones and laptops, without compromising the quality or speed of the video experience. From this, we could estimate the positions of both eyes and both shoulders and use some simple maths to calculate whether or not a candidate is slouching (forwards/backwards/left/right) with reasonable accuracy. In case that verbal description wasn't visual enough, here's a picture of posenet in action, with the keypoints mapped onto a frame of my head and shoulders:

<Image
  alt={`Posenet in action, with the keypoints mapped onto a frame of my head and shoulders`}
  src={`/static/images/interview-toast/posenet.png`}
  width={1200}
  height={725}
/>

The following diagram explains how we used the angles of displacement from the horizontal axis and the distances between key points from the camera to quantify slouching:

<Image
  alt={`How we used the angles of displacement from the horizontal axis and the distances between key points from the camera to quantify slouching`}
  src={`/static/images/interview-toast/posture-detection-math.png`}
  width={1200}
  height={725}
/>

## Want to find out more?

If you want to know more about this project I would recommend you check it out either at interviewtoast.com or check out the source code directly on [GitHub - interviewtoast](https://github.com/yp717/interviewtoast).


Everything you need to prep and make the most of your next interview.

InterviewToast


<Image
  src={`/static/images/digitalfootprint-earth/hero.png`}
  alt={`Digitalfootprint.earth blog post hero`}
  width={1920}
  height={1080}
  priority
/>

DigitalFootprint.earth is a project I started working on together with [@samlarsendisney](https://twitter.com/samlarsendisney) for the Esri X TechCrunch Hack for a Sustainable Future.

Esri and TechCrunch called on developers around the world to imagine new approaches to the most pressing global challenges, from the climate crisis to COVID-19. All participants were given free access to Esri's [ArcGIS platform](https://www.youtube.com/watch?v=OfX8MIvapT0) to biuld their projects and the winners were announced at TechCrunch Disrupt's 2021 conference.

As web developers, we care deeply about the web and the impact it has on society. We wanted to make this hack about a sustainable future personal.

By trying to improve the performance of our personal sites by peaking through lighthouse audits, we realised far more data was transferred when a page loads than we thought. Every byte of energy sent requires energy, which in turn produces CO2 and other greenhouse gases. With billions of people and pages on the web it became clear, that the tool we love the most ~the internet~ was doing a lot of harm to our planet.

According to the BBC, the internet is responsible for [nearly 3.7% of all global CO2 emissions](https://www.bbc.com/future/article/20200305-why-your-internet-habits-are-not-as-clean-as-you-think). Another study by the Boston Consulting Group (BCG) estimates that the internet produces almost [1 billion tonnes of greenhouse gases each year](https://www.sciencefocus.com/science/what-is-the-carbon-footprint-of-the-internet/). The scale of this problem is scary because when you think twice about these numbers, its clear that the environmental impact of the internet parallels that of plastic. The cloud where we store our data has a physical presence in thousands of data centres around the world.

To make things worse, the internet isn't put under nearly the same amount of scrutiny or the same standards as continue to try to achieve the United Nation's 2025 Sustainability Goals. We can do a lot better.

## What does it do?

Our project submission, DigitalFootprint.earth, uses mathematics and ESRI's mapping APIs to estimate and visualize the carbon footprint of the web. We split our project into three parts:

1. _Story Telling_: By using ESRI's ArcGIS API, DigitalFootprint.earth takes seamingly meaningless data and translates it into a tangible story that shows how serious the impact of the internet can be on our planet.
2. _Investigation_: DigitalFootprint.earth investigates a site's hosting, CDN, performance and size to suggest improvement mechanisms and simple steps to make the internet a greener place.
3. _Auditing_: By using DigitalFootprint.earth's web application and API, developers can consistently monitor and improve their site's performance, CDN response time, and other metadata via our scoring system. We also created a custom server-side-generated badge that can be displayed on a website to demonstrate to users that the developers are holding themselves accountable to ensuring their site is as green as can be.

This project was built over three weeks and there are a number of assumptions in the calculations though they provide a good starting point for any investigations!

<Image
  alt={`Screenshot of DigitalFootprint.earth homepage`}
  src={`/static/images/digitalfootprint-earth/DigitalFootprintEarthHome.png`}
  width={1200}
  height={785}
  priority
/>

## Challenges

We ran into a number of challenges during this hackathon but I think that's part of what makes the experience so memorable - transforming an idea from something we thought to be impossible into a product people could actually use and maybe even benefit from! Some of these main challenges were:

- _Working with a brand new API_: Esri's ArcGIS API was new to both of us and it was incredibly extensive. With such a large feature set things felt overwhelming at first and some parts of the API such as the service area API proved to be particularly tricky to get our heads around.
- _Using ArcGIS with Gatsby_: We wanted to use a framework like Gatsby or Next for this project (we used Gatsby for this one!) and the SSR that it utilises didn't work nicely out of the box with the ArcGIS maps. This meant we had to make some interesting modifications to lazy load in the map and use framer motion to make the transition more graceful.
- _Generating badges on the server_: It's not the most immediately obviously complicated task but generating images on a server - with out the DOM - was not easy because of how the HTML had to dynamically update based on the request parameters.
- _Data sourcing_: Collecting data to substantiate a lot of the calculations, insights and findings and to integrate these with our tools was one of the biggest challenges.

## TechCrunch

We got the opportunity to present this project at [TechCrunch Disrupt 2021](https://techcrunch.com/2021/09/18/users-guide-to-techcrunch-disrupt-2021) on the main stage in the finale of the competition! The image below is a screenshot from the moment when we found out that, to my disbelief, our project won the competition and took home the $10,000 prize and got the opportunity to present at Esri's Developer summit in 2022 too!

<Image
  alt={`TechCrunch Disrupt Conference`}
  src={`/static/images/digitalfootprint-earth/TechCrunchDisrupt21.png`}
  width={1200}
  height={671}
  priority
/>

## BONUS: What else can I do today to reduce my digital footprint on the web?

If you don't host your own site, that doesn't mean that there's nothing you can do to help reduce your carbon footprint through your internet usage. Here's a short list of some other small steps you can take that can go a long way toward helping our planet:

<Image
  alt={`Screenshot of DigitalFootprint.earth homepage`}
  src={`/static/images/digitalfootprint-earth/DigitalFootprintWorldBiggestSites.png`}
  width={1200}
  height={634}
  priority
/>

- _Keep your hardware longer_: the devices you access the internet through, whether its your laptop, phone or tablet have a heavy environmental footprint both through their manufacturing processes, day-to-day use and disposal.
- _Delete your old emails_: storing large amounts of data can also increase your digital footprints. Deleting those emails you haven't looked at or used for years can go a long way toward reducing the impact you have on the environment
- _Unsubscribe_: Email subscriptions and mailing lists produce a lot of CO2. Unsubscribing from the email lists that you don't read instead of letting them pile up in your inbox is another quick step you can take to be a greener citizen of the internet

## Try it out today!

At the time of publishing this post, DigitalFootprint.earth is still live. Why not try it out today by clicking [here](https://DigitalFootprint.earth)

You can also read more about the project in detail on the [DevPost submission](https://devpost.com/software/digitalfootprint-earth)!


Brings the impact you have when using the internet to life through interactive storytelling using maps

DigitalFootprint.earth

A Tailwind CSS plugin that adds the Radix Colors palette.

Using ML to develop models to create patient risk scores for COVID-19 for a clinical decision support system in the NHS called EPiC IMPOC.

tailwindcss-radix-colors-plugin

  
<Image
  alt={`Facebook`}
  src={`/static/images/covid-ml-epic-impoc/hero.png`}
  width={1200}
  height={550}
  priority
/>

Between my third and fourth year of university we had to complete a placement project for a company in industry. In my case, we were fortunate enough to be offered the opportunity to apply to self propose a project in conjunction with the Imperial College NHS Trust Hospitals and the Centre of BioInspired Technology at Imperial College London.

In this project, we used machine learning to develop models to create patient risk scores for COVID-19 and for probabilistic inference for a clinical decision support system in the NHS called EPiC IMPOC run by the Centre of BioInspired Technology at Imperial College London.

## Where did the need for this project come from?

The first case of COVID-19 was detected in Wuhan, China on the 31st of December, 2019. Within three months, the World Health Organization (WHO) characterized the viral disease as a pandemic because of its high reproduction rate and threat to global public health. With the UK's ever-increasing death toll and number infected, the government imposed a lockdown, shutting down all non-essential services on the 23rd of March 2020. Although restrictions across the globe are lifting, the UK and many other countries continue efforts to limit the spread of the virus and protect the National Health Service (NHS) from being overwhelmed.

According to the NHS, as of Saturday, the 23rd of May 2020, there have been 257,154 total lab-confirmed cases, 36,675 total associated deaths. Each day, the NHS estimates that there are another 2,959 daily lab-confirmed cases and another 282 daily associated deaths across the UK. The NHS reports the number of daily cases and associated deaths continues to decline. However, the implications of easing lock-down restrictions is important to consider in preventing the spread or resurgence of the virus.

There quickly came a crucial need for technical solutions such as clinical decision support systems to improve patient diagnosis, risk assessment and the distribution of medical resources for infection management.

## What is a Clinical Decision Support System?

A clinical decision support system (CDSS) is intended to improve healthcare delivery by enhancing medical decisions with targeted clinical knowledge, patient information, and other health information. CDSSs are composed of two main categories, knowledge based and non-knowledge based. In short, the knowledge based CDSS a predefined set of rules is created based on evidence from literature, practice, or patients. Alternatively, the non-knowledge based CDSS relies on artificial intelligence (AI), machine learning, or statistical pattern recognition.

## What did we build?

EPIC IMPOC (Enhanced, Personalised and Integrated Care for Infection Management at the Point of Care) is a clinical decision support system used in the NHS to improve the management of infectious diseases by facilitating data collection, infection diagnostics and antimicrobial therapy advice at the point of care.

In this project, we built a new module for EPIC IMPOC that focused specifically on integrating then current medical knowledge of COVID-19 and patient data with an inference engine to predict the likelihood of infection, the severity of infection, and to allow clinicians to retrieve the most similar patients to an individual patient at the point of care. Additionally, the patient retrieval system is packaged in a mobile and desktop friendly panel for clinicians within EPIC IMPOC to identify patients with similar severity levels, symptoms, and medical histories in the UK.

This EPIC IMPOC module was one of the first to undergo clinical trials in the NHS, starting in early July 2020. This solution will be a crucial part of the Imperial College Healthcare NHS Trust’s COVID-19 response to the second wave in the pandemic. The Trust’s five hospitals currently treat more than 1.125 million patients and employ 11,800 people annually.

Implementing a clinical decision support system for COVID-19 comes with a unique set of technical, social, and medical challenges. Mainly, the technical approach must adapt to the rapidly changing data and medical knowledge available. Furthermore, limited data creates challenges in developing robust machine learning algorithms to provide advanced, intelligent decision support for clinicians at the point of care. This required extensive research and testing of different algorithmic approaches, tuning their hyper-parameters and evaluating their limitations and strengths. The following sections of this report will include a literature review, the design criteria, a system architecture description, as well as a breakdown of the core components used in this module to overcome these challenges.

## System Architecture

The module performs two functions: the retrieval of similar cases and the risk assessment of patients in terms of COVID-19 infection.

A variety of technologies were used in the development of this CDSS module including but not limited to Python, Sci-kit learn, Pandas, MatplotLib, NumPy, LightGBM, XGBoost, HTML, CSS, JavaScript, Django.

The back-end was developed with a Python framework called Django, which is commonly used as a back-end for web-app development. The Django module interacts with the user via a front-end, developed with HTML, CSS and JavaScript.

The requests from the user via the front-end are then used to either query a database or are evaluated using a Machine Learning-based back-end within the framework. The back-end maintains different views for interactions with different algorithms and features of the module. These views include, for example, a view for the application of probabilistic inference as well as a view that allows clinicians to retrieve similar patients based on the patients they select from the view.

When the machine learning module is called, the pipelines developed in the research phase of this project are applied to the data, which is then returned via the Django framework to be rendered on the front-end.

The end-user, which is likely to be a clinician or medical care worker, interacts only with the front-end of the system. The front-end encapsulates all of the complexity of the system to present the information and functionality to the end-user in the simplest possible way via a desktop and mobile responsive panel.

The User Interface (UI) was developed using CSS, HTML and JavaScript. The front-end has the ability to allow clinicians to customise and choose the way they want to control, view and input data. This includes the ability to fill out forms manually, upload large spreadsheets of cases, query databases for existing patients, and perform risk assessments for patients in terms of COVID-19 infection. The abstraction provided by the front-end removes the complexity of data pre-processing, machine learning and database interactions, to streamline the experience for medical care workers.

### Risk Assessment Using Probabilistic Inference

The probabilistic inference (PI) function estimates the probability of COVID-19 infection in a patient based on key pathology biomarkers. It was trained and tested on a total of 1186 pathology (Table below). New data can be input by form, existing patient records, or by uploading a CSV file with several patients’ data. Three machine learning algorithms among other tested algorithms, each giving a different result, are used: LightGBM, Random Forest Classifier and Decision Tree Classifier. If a form or existing patient record is used, three different estimates are returned. In the case of a CSV file, an updated file with three new columns is returned, each column corresponding to a different estimator.

The following table summarises the class distribution for the important datasets used in the Probabilistic Inference Module

| Data               | Total | Covid-19 positive | Covid-19 negative |
| ------------------ | ----- | ----------------- | ----------------- |
| Pathology          | 77351 | -                 | -                 |
| Labels             | 1675  | 976               | 699               |
| Pathology ∩ Labels | 1186  | 758               | 428               |

### Patient Retrieval - Case Based Reasoning

The CBR system retrieves similar patients and provide them to the clinicians to support them with treatment prescription. The patients’ retrieval is going to be done using the pathology dataset which contains 837 pathology profiles of and the vital signs dataset which contains the vital signs, symptoms, radiology results and medical history of 262 patients (in the table below). For the former, a supervised and an unsupervised method were used, while for the latter only the supervised methodology was followed.

In the supervised approach, the weighted distance between a given patient and the remainder of cases is computed and the most similar cases are selected. These weights are pre-computed using each feature’s importance from the Random Forest. The system's performance was evaluated using the patient’s outcome to verify and investigate patient similarities. Different distance metrics, filters, imputers and scalers were tested and selected to maximise the performance.

On another hand, the unsupervised approach for CBR, uses an HDBSCAN algorithm initialised with the same pre-computed weights to cluster similar patients by their biomarkers and symptoms. The primary advantage of using a clustering algorithm is that it does not require re-sampling, imputation, and most importantly, labels.

The following table shows the class distribution for the important datasets used in the CBR Module.

| Data        | Total | Admitted to the ICU | Not Admitted to the ICU | Died | Survived |
| ----------- | ----- | ------------------- | ----------------------- | ---- | -------- |
| Pathology   | 837   | 443                 | 394                     | 517  | 214      |
| Vital signs | 262   | 31                  | 221                     | 60   | 202      |

Aside from the extensive functionality provided by the module, another key component of this system is the development of an intuitive app that clinicians can easily interact with at the point of care. The figure depicts a high level view of the system architecture. The following sections investigate the development of each component of the system and explain their inner workings in more detail.

<Image
  alt={'System Architecture Diagram'}
  src={'/static/images/covid-ml-epic-impoc/systemArchitecture.png'}
  width={1200}
  height={430}
  priority
/>


Probabilistic Inference using Machine Learning for COVID

Machine Learning for COVID Detection


<Image
  alt={`Python and F# Logos side by side`}
  src={`/static/images/funcypy/hero.png`}
  width={1200}
  height={550}
  priority
/>

An interpreter for a tiny functional language with a friendly python-like syntax to make functional programming easier to learn and understand

This project was quite different from anything I worked on before. The objective was to write an interpreter for a tiny functional programming language.

We called it FuncyPy because the idea was to use a friendly, python-like syntax for a functional programming language. All parts of the interpreter were written in fsharp. The idea was to make the syntax as simple as possible, so, as an example, here's what a simple factorial function would look like:

```fsharp
def factorial x:
    if (x==0):
        1
    else:
        x * (factorial (x-1))

y = 3
y + (factorial 4)

>> 27
```

I worked primarily on developing the parser and integrating the parser with the lexer and combinator runtime. This meant writing lexer, parser and end-to-end unit tests to find edge cases and improve the code's ability to handle errors.

The parser architecture I think is quite interesting. It's a demonstration of how powerful functional programming concepts can be because of how extendable the language is. The development pattern used uses smaller building blocks to compose progressively more complicated Parsers that we combined to parse the tokenized input into the abstract syntax tree (AST).

There's definitely a lot that could be improved about this design, from the error handling to the efficiency of some of the implementations, as well as some repetitiveness in the code. However, given that this was my first real exposure to functional programming, parsing and the functional paradigm, I really enjoyed writing this parser. This project is the culmination of only 3 months of functional programming and 3 months of F#. F# has come with a lot of challenges, particularly the documentation, which I think is often quite frustrating and poor. However, its easy-to-use maps, computation expressions and parser combinators make it a versatile language that I enjoyed working with.

## Side Note

FSharp's Computation expressions are one of my favorite parts of the language. They make it really easy to build much more complex parsers, while using a standard functional programming pattern and making the functions we spend so long working through before really useful.

## Want to know more?

I tried to keep this post brief. If you're interested in finding out more about the project, don't hesitate to reach out somewhere in the footer. The git repository has to be kept private for now because this project was part of a course at Imperial in 2020, and publishing it may give some students an unfair advantage and goes against department guidelines. I'm happy to discuss any aspects of the project if you're interested in more of the details or building something similar.


Built using F# over 3 months

Interpreter for a tiny functional programming language with Python-like syntax

FuncyPy


<Image
  alt={`IC Hack Winning Photo`}
  src={`/static/images/rfid-supermarket-checkout/hero.png`}
  width={1200}
  height={725}
  priority
/>

During my second year at Imperial, we were tasked with a second year project to design a product that ‘solves a problem’. The actual brief was quite vague and when we started, I never thought this was the project we would end up working on.

We began planning this project in early October. Six months later, we developed an end-to-end instant, reliable, and secure super-market checkout system using UHF RFID technology. We decided to kick off our project at IC Hack-2019 and built the first working prototype in under 24 hours, winning the DocSoc’s choice award for “Best Overall Hack.”

After the initial prototype was developed at IC Hack, the system was redesigned according to a new specification with metal beams, and the antenna array positioning was improved based on further research.

In case any of this is a bit hard to picture, here’s a CAD visualisation of what the system actually looked like:

<Image
  alt={`Python and F# Logos side by side`}
  src={`/static/images/rfid-supermarket-checkout/rfid-cad-model.png`}
  width={1200}
  height={725}
  priority
/>

In this post, I’ll provide a rough overview of how we put this together and try to explain how it works in a bit more detail.

## What is UHF RFID?

UHF RFID is an acronym for “Ultra High-Frequency Radio-Frequency identification.” It refers to a technology whereby digital data, encoded in tags (consisting of an integrated circuit and an antenna), is captured by a reader via radio waves. The barcodes you might have seen on supermarket items before encode digital data in a similar way, but use a visual collection of carefully placed bars instead.

However, barcodes have to be identified by a visual scanner that requires a line of sight. UHF RFID have several advantages over systems that use bar-codes, but most notably the tag data can be read outside the line-of-sight and from large distances (even up to 500m).

## Technology and Engineering

A range of technologies were used to develop both the initial prototype and the final version of our system. The primary objective of the project became to improve customer’s shopping experiences. We achieved this by getting the customer out of the store faster, without the long queue time and stress associated with traditional checkout systems. The project was split into separate subsystems and used a range of technologies, which the following sections provide a bit more detail about.

<Image
  alt={`System Level Schematic`}
  src={`/static/images/rfid-supermarket-checkout/system-schematic.png`}
  width={1200}
  height={725}
  priority
/>

### Physical System

Supermarkets tend to have multiple check out gates in parallel and we aimed for a traditional modular design where multiple units could be placed side by side. By using a strong, lightweight aluminium for the frame, we also had flexibility in the design that allowed us to hide wiring and optimise the antenna placement.

The functional operation of the physical system used C++, Raspberry Pis, ultrasonic sensors, laser cutting, 3D printing for motion detection, the physical metal structure, gate system and antenna mounts

### Facial Recognition

By integrating facial recognition, we were able to speed up the payment phase of the checkout experience even further, by recognising clients in a matter of seconds. Given the short amount of time we had for this project and the level of experience we had at the time, we used Open CV for the pre-detection stage and used Microsoft’s Azure AI for customer recognition for the core facial recognition functionality.

### Web Application

The checkout software was designed with ease of use in mind. We tried hard to keep it as simple as possible to minimise adaptation time for all consumer groups. The app was built using jQuery, HTML, CSS, and ElectronJS to provide a robust framework for developing the friendly user interface and it looked a little like in the photo below.

<Image
  alt={`RFID Checkout ElectronJS App screen`}
  src={`/static/images/rfid-supermarket-checkout/app-screen.png`}
  width={1200}
  height={725}
  priority
/>

### Backend

We used Java and SQLite for the backend, which interfaced with the antenna arrays and NodeJS for the server that interfaced with the Java backend and the web application.

## Want to know more?

I tried to keep this post brief. If you’re interested in finding out more about the project, don’t hesitate to reach out. I’d be happy to discuss any aspects of the project if you’re interested in building something similar, or are just interested in more of the details.


IC Hack 2019 1st Place Winner

An autonomous RFID Supermarket Checkout system built with UHF RFID

RFID Supermarket Checkout


<Image
  alt={`Design Rant Logo`}
  src={`/static/images/design-rant/designrant.png`}
  width={1200}
  height={550}
  priority
/>

_Note: DesignRant is currently no longer maintained and the site is not live.
You can still check out the code on [GitHub](https://github.com/DesignRant)
and please get in touch if you would like to get involved with maintaining
this project!_

DesignRant was a side project built over a couple of weeks with [Sam Larsen Disney](https://sld.codes) and [Ryan Gregory](https://ryangregory.dev). The premise was simple: _Short, sharp, user experience complaints_.

It’s exactly what it sounds like: short - capped at 1000 word - rants about our experiences on the internet. What started as a place for us to rant about questionable user experience decisions, quickly evolved into a much larger project. We had a vision to make it the perfect place for young UX designers and developers to share their rants, their critiques, and their visions for the user experiences and user interfaces that live on the web.

The project now lives as an open source organization on GitHub that will continue to grow and the site will officially launch later this week, with more than 300 sign ups before the site even launched.

A key element of DesignRant - the caveat - is that while many of the posts are often critical, and sometimes even controversial, you can be as negative as you like so long as you propose a solution.

How does the old adage go? It's easy to complain…but it's a lot harder to pick out the limitations of current designs and propose alternatives that could potentially improve experiences for users. I think it was something like that. 🤔

This will hopefully help with making our users and readers more aware of the importance of considering usability, accessibility and design at every step of the development process.

The large number of reader sign-ups on the site before it even launched, suggests it will be a great place for developers and designers to get exposure.

Every author has an author page and bio with links to their social media and a place to “buy them a coffee.” If this sounds like you or you know someone who might be interested, please do share and get involved.


Collaborative blog site for user experience rants

DesignRant is a blog site made for developers and designers alike to share short, sharp, user experience complaints.

Projects