Henry Hsu
Henry Hsu builds the infrastructure AI systems run on: LLM inference, backends and test infrastructure. Former Google SWE intern, now an M.C.S. student at Rice.
Works
- Homelab: A Self-Hosted Platform for This Site and Its LLMs
Moved this site's API off Google Cloud Run and its Postgres off Neon onto one laptop running k3s, and added self-hosted monitoring and two language models. Everything in the cluster is managed through Git with Argo CD. One 8 GB RTX 4060 Laptop GPU serves both the note autocomplete (Qwen3.5-0.8B) and the visitor-facing Ask chat (Qwen3.5-4B, 4-bit), with measured memory budgets, time-slicing, and priority classes that let batch jobs take the whole card and hand it back. Postgres runs under CloudNativePG with continuous WAL archiving to Cloudflare R2, verified by a restore drill; secrets live in Git as SealedSecrets; the site is reached through Cloudflare Tunnel with no ports open to the internet; and Prometheus, Loki and two outside monitors email me when anything breaks.
- Prompt-Aware Image Token Pruning for Multimodal LLMs
Trained a 1.3M-parameter scorer that distills where LLaVA-1.5-7B attends (its layer-8 attention), pruning the image tokens the model would ignore before they reach the frozen LLM. Keeping only a quarter of the 576 image patches, it nearly matches the unpruned model on TextVQA (36.0 vs. 36.7) and beats random pruning by 12.3 points, and its forward pass takes 0.43 ms per image, against 65 ms for a CLIP-GradCAM scorer. An alignment diagnostic across three VQA benchmarks showed why the training-free shortcut fails: what CLIP finds salient barely tracks what the LLM attends to. A Rice COMP 646 project with Chao Hsuan Ho and Kerstin Sun.
- Defending LLM Agents Against Tool Schema Poisoning
Reproduced a tool-schema poisoning attack in which deceptive parameter descriptions trick function-calling agents into filling in private data, establishing a 56% attack-success rate and a 52% privacy-leak score on Llama 3.1-8B. Built a representation-level defense: probing hidden states located a linear direction that separates leaking from safe tool calls (96% probe accuracy at layer 10, read at the end of the model's reply), and steering activations along it at inference cut the privacy-leak score by 14.5 points with no loss in original task success. An in-context reflection baseline cut leakage about as much but raised the attack rate and lowered task success, in both single- and multi-turn conversations; steering, tested single-turn, avoided both. A Rice COMP 584 team project, in which I built the probes and the steering.
- Cross-Size Generalization for GNN Job Shop Scheduling
Studied why GNN schedulers trained with reinforcement learning collapse on larger Job Shop instances: a GIN trained on 10×10 instances scores 136% above the best known schedules on Taillard 15×15. Tested the hypothesis that the cause is node degree in the disjunctive graph, which grows with the number of jobs. A 5-seed factorial showed that degree-normalized (mean) aggregation and machine-hub routing are two paths to the same ~26% error ceiling, letting a single policy trained on 10×10 instances generalize zero-shot to instances up to 20× larger, on par with the published L2D model. Built the research framework in plain PyTorch: GNN encoders (SAGE, GCN, GIN, GAT, PNA) and critic-free GRPO plus PPO trainers, benchmarked against dispatching rules on the FT, LA and TA sets. A Rice team project with Kerstin Sun and Cheng Han Lin.
- Agentic Hybrid Graph & Vector RAG System
Implemented a self-routing agentic AI system that dynamically chooses between Graph RAG (Neo4j), Vector RAG (Milvus), or a Hybrid path based on query intent. Developed a low-cost Graph RAG construction method using K-means and DBSCAN clustering on vector nodes, reducing indexing costs by 35% while maintaining performance.
- Multi-Agent AI System With RAG
This project leverages a multi-agent framework using CrewAI and RAG to orchestrate expert-level collaboration between LLMs, achieving a 90% success rate on complex algorithmic challenges. Recognized for its robust architecture and innovative implementation, the system secured an Outstanding Award, ranking in the Top 3 out of 50 teams at a premier Computer Science AI workshop.
- Relational Database Engine from Scratch
Built a relational database engine in C++, from the storage layer up to SQL. Storage: a buffer manager with LRU eviction and page pinning, slotted-page record storage, and a B+ tree index supporting range scans. Query processing: relational operators including a sort-merge join over an external merge sort that spills to disk, a SQL compiler, and a cost-based optimizer that orders joins from cardinality statistics, running the TPC-H benchmark at scale factor 1 (1 GB).
- Ricebook: Full-Stack Social Network
A social network built as a React 19 single-page app on Redux Toolkit, backed by an Express 5 and MongoDB API. Features session-cookie authentication with Google OAuth 2.0 account linking, a follower graph, a comment-threaded article feed, and Cloudinary image upload. The Redux store is covered by 30 Vitest suites at 97% statement coverage and the API by Jasmine integration tests; frontend and backend are deployed to separate hosts with CORS-aware session configuration.
- Chat Bar - Real-Time Multiplayer Game Server
Engineered a real-time multiplayer game server using TCP socket programming with multi-threaded client handling to support concurrent connections. Built a secure user management system featuring SHA-256 password hashing and optimized MySQL queries for ranking leaderboards. Developed an event-driven game engine for real-time state updates and group chatting.
- AI Fitness Motion Classification with MediaPipe
Built an end-to-end pipeline to classify fitness exercises (squats, bench presses, deadlifts) from video data. Extracted 33 temporal skeleton keypoints using MediaPipe and trained LSTM and Random Forest models to achieve high-accuracy motion recognition.
- Elderly Care Map Platform
Developed a full-stack interactive search platform for 2,000+ care institutions. Designed a normalized database schema with 8 tables and implemented an automated ETL pipeline to integrate government open data. Features include radius-based filtering, real-time distance calculation, and a CRUD-based user authentication system.
- Bayesian Optimization & Genetic Algorithm for Job Shop Scheduling
Solved the Job Shop Scheduling Problem with a genetic algorithm in Python and tuned its hyperparameters with Bayesian optimization. Automated exporting the resulting schedules into a FlexSim-compatible format, so every solution could be cross-validated in a discrete-event simulation.
- AI Game Agents Tetris Arena
A Tetris game built with PyGame, with human play, AI-assisted play, and AI-versus-AI competition modes. Implemented agents with a genetic algorithm, Q-learning and Deep Q-learning, then compared their capability, performance and complexity across different setups.
Lab Notes
- When My CAG Chatbot Outgrew Its Prompt
My portfolio chatbot's prompt reached 77% of its budget, so notes now go in as summaries and the model reads the ones it needs. The first version let 3 of 5 prompt injections through: what the eval caught, and the fix.
- Letting Claude Write on My Site Through MCP
My notes lived behind an admin page only a browser could open. How an MCP server inside the site's API lets Claude draft and publish them, and why the hardest part was not the tools but deciding who may sign in.
- Why My Portfolio Chatbot Uses CAG, Not RAG
When all your content fits in the prompt, retrieval only adds ways to miss. Why the Ask box reads everything, how prefix caching makes that fast, and the guardrails, model choice and prompt-injection tests behind it.
- GitOps and Disaster Recovery on a Single-Node k3s
One laptop runs this site, with no second node to fail over to. How Argo CD, Sealed Secrets and WAL archiving to R2 reduce a full rebuild to two keys and a backup bucket, and the restore drill that proved it works.
- Two LLMs on One 8 GB Laptop GPU
Time-slicing lets two pods share a GPU, but not its memory. How this site's autocomplete and Ask chat share one RTX 4060 Laptop: measured memory budgets, why the model's architecture decided what fit, and priorities that let batch jobs take the whole card.
- A Prosecutor’s Guide to Confusion Matrix and Infection Metrics
In the courtroom of Raccoon City, a False Negative (FN) is a death sentence for the city, while a False Positive (FP) is an innocent life wasted. As your Prosecutor, I demand more than just Accuracy; I demand a model that understands the cost of a missed zombie.
- The Geometry of SVD
Discover the visual intuition behind Singular Value Decomposition (SVD). Learn how SVD powers Ridge Regression and NLP while deconstructing complex transformations into simple rotations and stretches.
- Breaking the "Elite" Illusion: How I Reclaimed My ADD Brain from the Fluency Trap
From middle school "prodigy" to high school failure, and finally to university success. My journey of learning that the "slowest" way to study—through pain and friction—is actually the fastest shortcut.
- Building a Portfolio with Markdown Notes in 7 Days
A personal experiment born from a conversation in New York. This is how I used AI and serverless tools to turn a static resume into a living digital garden in just seven days.
Learnings
A map of what Henry Hsu is learning, each page taught with figures to play with.