memd — Prompt History¶

All structured prompts from this conversation, in order¶

Prompt 1 — Initial Situation and Goals¶

About Me: Ayush, a drop year student
Portfolio: ahhyoushh.github.io

Objective: Looking to make projects that will land an internship in research
labs, by cold mailing in interested fields. Targeting universities abroad.
Unclear about field to choose, not able to find research labs, and what's
the best in long term.

Interested fields:

Field: IoT, Avionics, Edge Avionics
Project idea: A UAV, making TX and receiver myself using ESP32 and SX modules,
LuckFox Pico Ultra or similar for object/entity tracking, making the PID
flight controller.
Fear: Too common, not much niche, not into aeronautics or mechanical engineering,
strictly ECE or CS. Also fears no good research labs will take a dropper.

Field: (less) EdgeAI, ML, Computer Vision
Project idea: A table viewer, views table from above, crops objects, generates
embeddings, semantic search through memory, using LuckFox or similar.
Fear: Too simple, not niche, not impressive, won't get upvotes, no credibility.

Job: Analyse situation, suggest better projects, which will get reach and be
interesting, not like a rover etc.

People I want to be like:
https://datavorous.github.io/
https://0xmukesh.github.io/

Prompt 2 — Clarifying Objective¶

I don't want foreign labs. I am applying to foreign unis after my gap year
so I need a research internship before that. Next fall 2027 I am applying for.

I find making a small inference runtime for YOLO or something, then using
that inference to make my table viewer the most interesting. I will also
like to build my own vector search library like spheni of datavorous too.

Give me a full plan, why you think fits the best, how to find research
papers and actually interesting research etc, how to become like Sagnik.

Prompt 3 — Does YOLO.C Already Exist?¶

I want to build inference library, quantisation library for some models.
Does yolo.c already exist?

Prompt 4 — Scoping the Project Stack¶

So I write my own inference, own quantisation and make my own CNN model?
We can skip vector search library then I guess or we can just brute force
and later implement?

Prompt 5 — ESP32 and NPU Question¶

If I could run it on ESP32 I really will want to. But isn't ESP32 CPU only?
What if I want to go into NPU?

Prompt 6 — Hardware Decision (Single Device)¶

What's the best hardware I should get? I only want to get one hardware for this.

Prompt 7 — Budget Expansion to 10k¶

Where do I buy from and what do I buy?
If I had say 10k budget what would be my options?

Prompt 8 — Coral USB Clarification¶

But the USB is not really a chip?

Prompt 9 — What Do Research Labs Use?¶

Research what is generally used in research labs?

Prompt 10 — Qualcomm and Lab Research¶

What about Qualcomm? Find me some research labs for me, tell me what they do,
what hardware they use, and pinpoint a good hardware for it.

Prompt 11 — ESP32 Inference Question¶

How about I do it on an ESP32, right inference for ESP32 too? Like a proper ESP32?

Prompt 12 — ESP32 Only Decision¶

Nah I can't find LuckFox, better do it all for ESP32. ESP32 is globally
recognised too, it will add weight to my posts.

Prompt 13 — Learning C and References¶

How do I learn C properly for this, like Beej's guide or something.
Use datavorous Reddit to find what he recommends, and what my prereqs for this will be:
https://www.reddit.com/user/Shonku_/
Output a references.md that will guide me to learn prerequisites.
Also timeline is messed up, its June 2 and applications start by Nov.

Prompt 14 — Is LuckFox Really the Best?¶

So is LuckFox really the best choice I've got?

Prompt 15 — Custom Inference vs NPU Clarity¶

So I won't use my custom inference on LuckFox for the actual NPU?

Prompt 16 — What Do Research Labs Actually Use?¶

Research what is generally used in research labs?

Prompt 17 — Qualcomm and Specific Labs¶

What about Qualcomm? Find me some research labs for me, tell me what they do,
what hardware they use, and pinpoint a good hardware for it.

Prompt 18 — Rewrite All MDs for ESP32 Only¶

Only ESP32, cheaper and better. Rewrite all my MDs for this plan.
Also add some fun research papers like Attention Is All You Need,
I'll blog them on Twitter etc.
Keep the MDs structured, simple to understand.
prompts.md should basically contain all structured prompts I gave you.

Prompt 19 — Audio Pivot Question¶

Instead of CV like desk viewer how about something like Shazam or audio classifier?

Prompt 20 — Life Memory Device (from ChatGPT conversation)¶

[Uploaded ChatGPT memory export]
This was a chat I had with ChatGPT. I wanna make this — a device that keeps
listening to me and makes a graph with memories or something.

Prompt 21 — Form Factor and Sensor Scope¶

I'll just skip the camera then? And add a motion sensor or something,
so I can keep this like a bracelet? Or just audio is fine too?

Prompt 22 — Final Direction¶

It's not really a bracelet, just call it a device.
Write all three MDs properly, with proper interesting trending research
like Attention Is All You Need and some with AGI and privacy like the
big techs keep talking about.

Final Decisions Summary¶

Device          : ESP32-S3-N16R8 + INMP441 I2S microphone
                  No camera, no motion sensor, audio only
                  ~₹1500-1750 total

Project         : Ambient Memory Device
                  Continuously captures audio events
                  Custom INT8 inference runtime in C (no ML frameworks)
                  Tiny CNN: log-mel spectrogram → event type + 32-dim embedding
                  Memory packets synced to local Python backend
                  Knowledge graph + cosine retrieval + optional local LLM

Research space  : TinyML + episodic memory + privacy-preserving edge AI
                  Intersection of MCUNet-style systems work and
                  MemGPT/Generative Agents-style memory research

Research angle  : "What is the minimum audio representation that preserves
                  sufficient semantic information for episodic memory
                  retrieval under severe hardware constraint?"

Target labs     : IIIT-H CVIT + Qualcomm Edge AI Lab (primary)
                  IISc DESE, IIT Madras ESB (secondary)

Timeline        : June → November 2026 applications
                  Blog Post 1: end of August (runtime + quantisation)
                  Blog Post 2: end of September (full system)
                  Cold mailing: September onwards

Form factor     : Desktop device (not wearable)
                  Breadboard prototype first, packaging later

Key People¶

Sagnik Bhattacharya (datavorous) https://datavorous.github.io/ IIIT-H. Qualcomm NPU SDK work. Built spheni (vector search in C), 2KB chess engine. Pattern: well-known problem + severe constraint + from scratch + good writeup = reach.

Mukesh (0xmukesh) https://0xmukesh.github.io/ Generalist across unusual intersections. Biotech, crypto, computational biology. Credibility from shipping things real ecosystems used.

The Narrative Through All Decisions¶

The project evolved through this conversation:

UAV with ESP32 TX/RX
    → too mechanical, not ECE/CS enough

Table viewer with LuckFox
    → interesting but obscure hardware, no NPU story without LuckFox

Table viewer on ESP32 only
    → globally recognised hardware, stronger constraint story

ESP32 + audio (Shazam)
    → better community recognition, but generic if just classification

Ambient memory device (audio only, ESP32-S3)
    → unique application framing, directly aligned with frontier AI research
    → privacy angle is genuine and timely
    → research question is real and open
    → demo is visceral and explainable to anyone

Last updated: June 2026