Esse

_

Envisioning the future of healthcare with artificial intelligence

Perspectives and objectives

Healthcare needs all the help it can get

The world's population is getting older, and with it the incidence of chronic illnesses is higher. Global healthcare systems won't be able to cope, as current solutions aren't enough.

Solution

Care from home

Esse is a pre-primary care solution that brings healthcare closer to the person. Esse takes care of someone by detecting untoward conditions and connecting to smart devices to modify the environment and facilitate recovery. Although current artificial intelligence (AI) models aren't as sophisticated yet, AI models in the future have the potential of realizing this concept.

Esse is a pre-primary care solution that brings healthcare closer to the person. Esse takes care of someone by detecting untoward conditions and connecting to smart devices to modify the environment and facilitate recovery. Although current artificial intelligence (AI) models aren't as sophisticated yet, AI models in the future have the potential of realizing this concept.

Esse is a pre-primary care solution that brings healthcare closer to the person. Esse takes care of someone by detecting untoward conditions and connecting to smart devices to modify the environment and facilitate recovery. Although current artificial intelligence (AI) models aren't as sophisticated yet, AI models in the future have the potential of realizing this concept.

Role

AI Strategy

Prototyping with Code

Project Type

Conceptual

Final project for my master's degree: Human Interaction and Artificial Intelligence

Duration

Dec 2023 to Jul 2024

Location

Barcelona 🇪🇸

PROCESS

PROCESS

The Process

Process

View solution

View solution

View solution

Landscape Study

1 of 3 adults have chronic illness being managed at home

Among these, most won't reach 70 years old. A majority of the population will be older than 65 by the year 2050. We all know what happened when the COVID-19 pandemic struck–people were stuck at home and a lot were left without anyone to take care of them.

A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.
A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.
A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.

By 2050, a quarter of the population will be older than 65 in Europe, China and Canada. The aging population will strain the healthcare system.

A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.
A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.
A collage of images including close-up shots of three people during a user interview, and external shots of Raffles Medical facilities.

When a person wants to understand their health better, they currently have a lot of options with varying degrees of comprehensiveness, automation and accessibility

A screenshot compilation of customer reviews from a teleconsultation app on a digital store, highlighting issues like long waiting times and connectivity problems with critical and negative feedback visible.
A screenshot compilation of customer reviews from a teleconsultation app on a digital store, highlighting issues like long waiting times and connectivity problems with critical and negative feedback visible.
A screenshot compilation of customer reviews from a teleconsultation app on a digital store, highlighting issues like long waiting times and connectivity problems with critical and negative feedback visible.

The iron triangle of healthcare: a solution will always compromise quality, access or cost

Jobs-To-Be-Done

Healthcare of today won't be enough for the needs of tomorrow

People want to stay healthy without having to worry so much about their health. I talked to nurses working in hospitals and retirement homes on common ways they manage their patients. The sick go to doctors and wait in long queues. Devices aid in collection of data but can't really check whether a person is well or not. AI can help healthcare become more comprehensive, automated and personalized.


There's a concept called the iron triangle of healthcare where a solution will always have compromises but with AI, I believe these can be tackled altogether.

People want to stay healthy without having to worry so much about their health. I talked to nurses working in hospitals and retirement homes on common ways they manage their patients. The sick go to doctors and wait in long queues. Devices aid in collection of data but can't really check whether a person is well or not. AI can help healthcare become more comprehensive, automated and personalized.


There's a concept called the iron triangle of healthcare where a solution will always have compromises but with AI, I believe these can be tackled altogether.

People want to stay healthy without having to worry so much about their health. I talked to nurses working in hospitals and retirement homes on common ways they manage their patients. The sick go to doctors and wait in long queues. Devices aid in collection of data but can't really check whether a person is well or not. AI can help healthcare become more comprehensive, automated and personalized.


There's a concept called the iron triangle of healthcare where a solution will always have compromises but with AI, I believe these can be tackled altogether.

Human-Computer Interaction

Building the right thing

From just being able to detect blood pressure via video through deep learning algorithms, I pushed my idea further to be a room that can detect different biomarkers to know a person's health status.


Knowing about one's health status isn't enough though, I thought of a smart mirror that can recommend health interventions after collecting health data.


The next idea was to have the AI work in the background, which would mean less friction, making interactions more seamless. This concept is called ambient computing.


Missing from current solutions is the technology not knowing if a person has recovered already. I remembered the nursing process back when I studied for my undergraduate degree. I implemented this process of cycle of care where the AI checks in on you every time to make sure a person is healthy.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

A mirror seemed personal and ubiquitous enough to gain traction but as I analyzed how a person initiates the interaction with a mirror–even standing in front of one would still be a friction point

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

ADOPIE: the nursing process

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Because of the sensitive nature of the information, those with datasets also don’t freely give them away.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

One of the APIs uses deep learning algorithms and remote photoplethysmography (rPPG) techniques to measure vitals signs data from video in a contactless manner.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

While I was looking for datasets for computer vision, the collections I saw always considered the Fitzpatrick scale. This schema describes a way to classify the skin by its reaction to exposure to sunlight. This inspired me for the colors of the visual identity and interface.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Adding "no yapping" at the end to prompts gave direct and concise answers. I gave specific prompts to the LLM so it can give the desired results.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

I had to balance between using few-shot prompting (giving samples) and shortening the prompt so that it doesn't use as many tokens

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Prompt chaining allowed the LLM to focus on one task and give an output which would then be fed to another prompt.

Designing with code

Building the thing right

I learned how to code with Python to present the idea in a more tangible way. I looked into datasets with parameters I needed and realized how there’s not a lot of available public video data with associated vital signs.


I decided to use APIs from pre-trained AI models for my prototype. To build it, I created a software architecture diagram first to help me break down the tasks on how the program will work.


I heavily used prompt engineering to adjust the results of OpenAI’s large language model (LLM). To better handle the tasks and optimize the output of the LLM, I broke down how the information was processed into chains of actions. Some call this technique prompt chaining, others call it agentic UX. Whatever it’s called, it helped me refine the results of the model.

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Laying out the software architecture helped in breaking down the tasks and code easily

SOLUTION

SOLUTION

The Solution

Solution

Proof-of-concept

Taking care of you. Always.

The tagline Taking care of you. Always. denotes omnipresence and is the main value proposition. Current health systems could not cater to all people all the time but with the help of artificial intelligence, it can augment the capacity of healthcare systems, serving as what I would call pre-primary care. This does not intend to replace healthcare staff but reducing their workload by first preventing illness and providing initial care right at a person’s home.


Esse empowers your health journey with real-time monitoring and personalized interventions. it does this by:

The tagline Taking care of you. Always. denotes omnipresence and is the main value proposition. Current health systems could not cater to all people all the time but with the help of artificial intelligence, it can augment the capacity of healthcare systems, serving as what I would call pre-primary care. This does not intend to replace healthcare staff but reducing their workload by first preventing illness and providing initial care right at a person’s home.


Esse empowers your health journey with real-time monitoring and personalized interventions. it does this by:

The tagline Taking care of you. Always. denotes omnipresence and is the main value proposition. Current health systems could not cater to all people all the time but with the help of artificial intelligence, it can augment the capacity of healthcare systems, serving as what I would call pre-primary care. This does not intend to replace healthcare staff but reducing their workload by first preventing illness and providing initial care right at a person’s home.


Esse empowers your health journey with real-time monitoring and personalized interventions. it does this by:

Side note: The visuals presented here have been redesigned to reflect my current design approach and may differ from the original.

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Assessment

Collecting objective data through connected smart devices and subjective data through voice input–all done in the background

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Diagnosis

Identifying concerns that need to be addressed after analyzing the data

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Outcome identification

Setting goals to be achieved based on the identified issues and diagnoses

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Planning

Recommending health interventions that a person could do

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Implementation

Modifying the environment through connected smart devices

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Implementation

If needed, contacts local emergency services

A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.
A screenshot showing the previous interface of a teleconsultation app, emphasizing a queue-based system and highlighting various interface problems such as a disorganized hierarchy and excessive prominence of rarely used features.

Evaluation

Checking if goals were met and continues checking for data to complete the cycle of care

Prototype Demo

Seeing it in action

I was able to showcase the final prototype through a video where I was watching TV while my phone camera was placed in front of me. In the future, this camera would be an unnoticeable sensor in the background recording data regularly. The video then shows how the prototype works, showing that I was healthy and gave me recommendations to maintain my health.

Subtitles are available to explain the demo

AI Architecture

Intelligent care

Esse will use AI throughout the whole process to enhance every step of healthcare, from assessment to evaluation. Collecting subjective data through Natural Language Processing (NLP) is crucial because it captures the patient’s personal experiences and symptoms, which are vital for accurate diagnosis and personalized care.


Medical Language Models (LLMs similar to Med-PaLM 2) help set precise health goals by analyzing complex medical data, ensuring tailored and effective interventions.


Robotics and automation are used to modify the environment, like adjusting lighting and temperature, to support optimal health conditions at home. Ideally, this AI operates directly on your device, keeping data secure and processing fast.


Additionally, converting records into digital twins offers a dynamic and comprehensive view of a patient’s health, allowing for more precise monitoring and treatment plans.

An image showing a cluttered teleconsultation app interface with the patient video off to the right side, a list of navigation tabs on the left, and multiple boxes with patient information and consultation details. The layout appears disorganized with poor information hierarchy.
An image showing a cluttered teleconsultation app interface with the patient video off to the right side, a list of navigation tabs on the left, and multiple boxes with patient information and consultation details. The layout appears disorganized with poor information hierarchy.
An image showing a cluttered teleconsultation app interface with the patient video off to the right side, a list of navigation tabs on the left, and multiple boxes with patient information and consultation details. The layout appears disorganized with poor information hierarchy.

This holistic approach ensures proactive and reactive care, making healthcare more efficient and personalized.

CONSIDERATIONS

CONSIDERATIONS

Considerations

Ethics & Guardrails

Righting the thing built

Artificial intelligence, at least in its current form, is known to commit a lot of mistakes and make things up. It does this while seeming confident and sure. This has serious implications in healthcare. Since Esse is dealing with lives, a lot of risk is involved.


AI is fed with and trained on data. Before the AI is released to production, a lot of quality checks and clinical validation has to be done to minimize mistakes.


Healthcare is a complex and multifaceted field, filled with challenges and considerations that need careful navigation.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Confusion matrix showing AI’s predictions that don’t match with reality and its effects on the user

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Once the AI is in the hands of people, safeguards are put in place and the user is given control. One of the safeguards is making sure a person understands how their sensitive personal data is collected, kept safe, and used.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

As the platform collects data, incentives for the user can be put in place to purchase them and train the AI better. This would keep the platform’s datasets up-to-date and not just be synthetic. Discounts on subscriptions can be offered and users’ data should not be used for training unless they give explicit consent.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

With neural networks and deep learning, AI models become an opaque black box and it is difficult to trace how the algorithm came up with its output. Measures should be implemented where it can provide clear, understandable justifications for their decisions and actions, making their inner workings transparent to users.

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

If the system still gets it wrong, the user can provide feedback to the platform as to what it got wrong and the AI learns from it. People are also working in the background, ensuring quality output and training the models with constant updates. 

This process is called human-in-the-loop. Human-in-the-loop refers to systems or processes where human intervention is integrated into the operation of automated systems. It involves human oversight, input, or decision-making at critical stages to enhance accuracy, address uncertainties, and ensure ethical considerations are met.

FUTURE

FUTURE

The Future

Looking forward

Esse seamlessly integrates into daily life, providing dedicated care in the background. The AI operates directly on your devices, ensuring data security and fast processing. This integration makes health monitoring feel like a natural part of your routine, addressing your needs without being intrusive.


Imagine a future where care technology is completely ambient, embedded throughout your community. Esse's systems not only enhance individual health but also contribute to making entire cities healthier. With these advancements, healthcare becomes a seamless part of everyday life, providing support and interventions exactly when needed. The images on the storyboard below were created through Midjourney–an image generation AI tool.

Healthcare is a complex and multifaceted field, filled with challenges and considerations that need careful navigation.


I would have wanted to do a lot more if I had the time and resources. I wanted to connect Apple’s HealthKit, which would mean I could have collected the health data from my iPhone and Apple Watch to provide a more holistic view of my health. I wasn't able to because I haven't learned how to build iOS apps just yet.


I also wanted to connect more APIs that use artificial intelligence to understand health better. Nowadays, voice analysis tools can determine a person's mental state. Medical LLMs are also better at reasoning and giving recommendations. Unfortunately, access to these APIs are closed for now.


To improve the prototype, I could also have used a cloud server to store videos instead of downloading video recordings to my local folder, this would have made the prototype a truly functional web application. 


More research needs to be done to see how people would perceive such a service. Market dynamics also have to be looked into further to understand the opportunity for growth. 

As with every other project, design is never done. It can always be iterated. These would be the immediate things I would do to improve Esse.

This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.
This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.
This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.

Care technology is a seamless part of daily life– health needs are addressed through them

Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.
Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.
Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.

Sensors can give accurate diagnoses, devices can heal symptoms and can predict conditions

This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.
This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.
This interface showcases a teleconsultation session with a patient, highlighting the patient's video front and center, and all relevant medical and personal information neatly organized around the video feed. This design focuses on usability, emphasizing the interaction between the doctor and the patient.

Everything is connected and only the most urgent cases will alert the healthcare team

Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.
Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.
Displays a teleconsultation scheduling interface, showing appointment times and details such as the patient’s name, reason for the consult, and status. This screen is organized to show a clear hierarchy of information, making it easy for healthcare providers to navigate and manage appointments.

As much as possible, care is brought to one's home instead of having to go to the hospital

REFLECTIONS

REFLECTIONS

Reflections

Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.
Annotated interface of a teleconsultation app highlighting flaws such as overly prominent search functions, confusing information hierarchy, and the redundancy of some tabs and features.

Presenting Esse

Looking back

I would have wanted to do a lot more if I had the time and resources. I wanted to connect Apple’s HealthKit, which would mean I could have collected the health data from my iPhone and Apple Watch to provide a more holistic view of my health. I also wanted to connect more APIs that use artificial intelligence to understand health better. Nowadays, voice analysis tools can determine a person's mental state. Medical LLMs are also better at reasoning and giving recommendations. Unfortunately, access to these APIs are closed for now.


To improve the prototype, I could have used a cloud server to store videos instead of downloading video recordings to my local folder, this would have made the prototype a truly functional web application. 


More research needs to be done to see how people would perceive such a service. Market dynamics also have to be looked into further to understand the opportunity for growth. 


As with every other project, design is never done and can always be iterated. These would be the immediate things I would do to improve Esse.

9.45 / 10

9.45 / 10

final course grade

final course grade

Designing for AI

Designing for AI requires adhering to key guidelines for effective and ethical solutions. First, understanding the data used to train AI models is crucial as it informs how the model will perform; it should be accurate, representative, and free of biases to ensure fair and reliable outcomes.


Since the user experience depends heavily on the model’s performance, designers should provide options for users to tweak outputs, give feedback, or disregard the AI altogether. Recognizing each model’s strengths and limitations helps in unlocking its potential to assist people effectively.


Ethics plays a vital role in AI design, encompassing privacy, transparency, and the impact of AI decisions. Ideally, AI should eliminate repetitive tasks, enabling users to focus on more meaningful, creative, or complex work.


Lastly, designers should aim to “hide” the technology by crafting seamless, intuitive experiences that allow users to interact naturally, without needing to understand the underlying AI mechanisms. This balance builds trust and fosters a user-centric approach to AI applications.

Next Project

Coins.ph Crypto Service

Growth design for the largest crypto platform in the Philippines