Andrea Gómez Jaime, a first-year MSc student in the Biological and Biomedical Engineering (BBME) program, recently participated in the inaugural MIT Hardware x AI Hackathon (Hard Mode) held at the MIT Media Lab from March 6–8. In this student spotlight, Andrea shares what inspired her to apply, the project her team developed during the competition, and the lessons she learned from the experience.
Q: Congratulations on your acceptance and participation in this hackathon! It must have been a really exciting opportunity. Can you tell us abit about what the MIT Hardware x AI Hackathon is[1] and what drew you to apply?
The Media Lab is well-established at MIT – it's an interdisciplinary lab with biomedical engineering, design, computer engineering, and other focuses – and this is the first time that they’ve done this hackathon competition. At McGill, I’m part of the Shared Reality Lab and the Biofluids and Global Health Lab, and this hackathon was related to both of these labs and the approaches we use. At Hard Mode, the organizers wanted to reframe what a normal hackathon is, and what we can use software and applications for. With how AI [artificial intelligence] has re-shifted the types of things that we can do with the tools that we have, and the amount of things that we can accomplish in a short time, the organizers wanted us to see how we could embed AI into the physical world. The purpose was to build a physical product and not just an interface – which is what drew me to apply. I thought this was a good challenge for me to reflect on how I could interact with AI outside of an app. Now that AI is out there, it’s not going to stop. We will need to learn how to use it and what we can do with it beyond applications and chat bots.
Q: What was the application process like?
I heard about it by chance. I’m a big fan of the work done at the MIT Media Lab, and while I was visiting a friend in Boston, I came across the opportunity. It was perfect timing. In the application process, I had an interview and had to submit my portfolio of all the projects that I’ve done across computer engineering, hardware, and wet lab-based projects. I also had to submit a short paragraph about which track I wanted to submit the project into, who I am, and what my interests are – this allowed for a pre-reflection of what I wanted to do in this competition and allowed me to reflect on how we could integrate AI into a new hardware and use it differently than how it’s already used.
Q: Can you briefly describe your research and how it connects to this competition? What project did you work on during the hackathon and which of the 6 Core Tracks[2] were you involved in?
I’m working in computer vision systems that can read lateral flow assays and then I’m also able to work in public health through AI. In the Shared Reality Lab, we are encouraged to have side projects that we are part of external to our thesis, while we are in the process of completing our degree. I’ve been very interested in the trend of smart patches and how we can record biomedical data and biosignals from our bodies, other than with smart watches. In the lab, I’m hoping to work on a side project where I’m interested in creating a smart tattoo!
Once I started building the team within the hackathon, I knew I wanted to do something outside of my grad school research, and still use data tracking and a dashboard approach, integrating vitals measurements.
Our team chose the Reflect track because many of our members had biomedical engineering backgrounds and wanted to use the data we collected in this competition for something other than health-related projects. We wanted something that would ground you in the moment, pulling you away from your screens and allowing you to be more present. Ultimately, we designed a music box that generates music based on how your day was.
We mapped this using heart rate sensors and galvanic skin response sensors, tracking changes in skin conductivity associated with micro-sweat droplets and sweat gland activity. The micro-sweat drops will be present when you’re excited or stressed, but when you’re calm or sad, their presence will be lower. Similarly, heart rate varies as our emotions change, becoming faster or slower if we are more excited or calmer. We used these signals to determine and track the user’s emotional state throughout the day. Then, we used an LSTM [Long Short-Term Memory network], which is designed to learn patterns across time, to allow the system to understand how the user’s mood was evolving during the day and translate those patterns into musical parameters, like melody and tempo. Based on this, the system generates music that plays when the user hand-cranks the arm of the music box. The idea is that instead of just pressing a play button, users actively create their music as a grounding exercise.
The music box is like a “Spotify Wrapped” using your vitals!
We really took our time conceptualizing this and it was great to work with people from other industries to see how they think and conceptualize ideas prior to creating the prototype.
Q: How did your project turn out, and what challenges did you face during the hackathon?
We didn’t win, but it was still really fun and I really enjoyed it! I hadn’t done a hackathon in a while, so I wanted to focus on having fun and experimenting with hardware, since I usually just code.
It was tough though – I worked on the sensor portion of the project, however, I haven’t played around with sensors in 3-4 years. I was working on connecting the heart rate sensor and galvanic skin response sensors and sending the data through the AI pipeline, which one of my group members also worked on creating. We didn’t connect to an API [Application Programming Interface] from a wearable, but instead we wanted to create the sensors ourselves to allow us to collect raw data and analyze it how we wanted to, in order to generate the music. Wearables can have too many limitations because they provide processed scores, rather than raw data.
We also built 2 hardware devices, the biosensors and the music box. It was difficult because it was a lot of work and working with biosensors is always challenging. There is a lot of noise, and you have to be very precise with them. It took longer than expected to extract the data that I wanted because I had to ensure that the sensors were accurate.
We used soldering in our design, but this was fragile, and the soldering station was located separately from our team station. Because of this, we had to ensure that we didn’t break it, adding another element of difficulty.
We then had to understand how to generate the music – we understood what the vitals were telling us, but how could we translate this into music? Since none of us had a background in music, we had to figure out these different metrics and translate certain moods to various tempos and sounds. We thankfully have some friends who are musicians and music engineers, so we called them and we had them give us a quick masterclass in how this could work and how it could create music.
Finally, we wanted to incorporate the manual process of cranking the music box’s arm as a means of hearing the music. There are no mechanics built in that do anything when someone cranks the arm, but instead sensors that initiate the music to play, simulating this action.
Q: What was it like working and collaborating in such an intensive, fast-paced environment?
It’s stressful for sure. I learned from past experiences that because it moves so fast you have to enter it with a schedule. We spent much time brainstorming and building our concept, but we couldn’t compromise on this. It was important that we created the concept and stuck to it. We had doubts that we had to talk through, but our main concept remained throughout and provided clarity for what we had to build.
We also had daylight savings time taking place on the weekend of the hackathon, which left people confused about timing and just added another hurdle.
We had adrenaline building this though; it was a lot of fun. The day before the deadline was the most stressful since the pitch is only 2 minutes long and we had to decide what we would say and the timing of this. Since I did 2 hackathons previously to this, I had experience, and I knew what I was getting myself into, which was important!
Q: What new skills or perspectives did you gain from participating?
I had never embedded AI into anything before – I have used models, but never put them anywhere, so that was cool to learn. I learned about hardware prototyping and using wearables with sensors as well... this was mostly a refresher of what I already knew, but I want to explore this in my master’s thesis too and it was good to review it. We also had people on our team from different levels of their studies and careers and different subject matter expertise – so that was very helpful in learning about how to conceptualize these kinds of designs and get varying perspectives.
Q: How do you see experiences like this shaping your future research or career direction?
I think it’s nice to spend time doing this. It seems school-related, but really, you have so much freedom to do what you want to do. You’re able to explore what you usually can’t inside the classroom. Biomedical engineering is focused on health outcomes, which drew me to it, but its nice to leverage opportunities to explore other ideas that you wouldn’t typically explore.
I was outside of my usual environment, and I got to learn about other areas that I didn’t know as much about or things that I put aside for a little while. It’s nice to see what other things you can do with the skills you have. When you graduate, you have to be flexible. You may not end up in the biomedical industry as an engineer, so it’s important to be able to apply your skills elsewhere.
Q: What advice would you give to other biomedical engineering and bioengineering trainees interested in applying to similar opportunities?
You should definitely do it! It’s a lot of work, but I don’t regret spending my reading week doing this. It’s a good opportunity to connect back to why you joined engineering in the beginning. Coming into the competition with a general idea of what you want to build or a concept saves you time and really makes a difference in the final product. Understanding peoples’ needs is also an important part of biomedical engineering; you’re doing things because they’re fun, but also because they have a purpose!
Thank you, Andrea, and best of luck with your future projects!
[1]From the MIT Hardware x AI Hackathon website: “HARD MODE is MIT's 48-hour hardware × AI hackathon focused on building the future of intelligent objects: devices that sense, learn, adapt, and respond to the people around them. The challenge is to imagine what else AI could be. Not another chatbot. Not another app. Real hardware you can hold, wear, share, install, and live with. 200 participants will integrate AI with physical systems to prototype tangible, functional artifacts. Systems that rethink how humans connect, learn, reflect, work, play, and thrive. Hosted by MIT Media Lab's AHA and the Design Intelligence Lab, HARD MODE brings together engineers, designers, and researchers to push the boundaries of what intelligent hardware can be.”
[2]From the MIT Hardware x AI Hackathon website: “PLAY: Create joyful hardware: interactive stories, hybrid games, AI-driven performance tools, and installations that spark wonder and shared play. LEARN: Reimagine how people learn with adaptive devices, generative study aids, and tools that turn abstract ideas into accessible, hands-on experiences. WORK: Invent AI-powered tools and processes that expand human capability: new mediums, generative hardware, and systems that help people create the impossible. CONNECT: Create tech that strengthens relationships and community-devices that reduce loneliness, support organizing, and help people find belonging. REFLECT: Design technologies that support contemplation and growth-tools for mindfulness, mood tracking, habit reflection, and understanding yourself. THRIVE: Build tech that boosts performance, care, and autonomy: from wearables and assistive tools to systems that help people live longer and better. ”