Works
Projects by Henry Hsu across GenAI systems, infrastructure and machine learning: what each built, how, and what it did.
- Homelab: A Self-Hosted Platform for This Site and Its LLMs
Moved this site's API off Google Cloud Run and its Postgres off Neon onto one laptop running k3s, and added self-hosted monitoring and two language models. Everything in the cluster is managed through Git with Argo CD. One 8 GB RTX 4060 Laptop GPU serves both the note autocomplete (Qwen3.5-0.8B) and the visitor-facing Ask chat (Qwen3.5-4B, 4-bit), with measured memory budgets, time-slicing, and priority classes that let batch jobs take the whole card and hand it back. Postgres runs under CloudNativePG with continuous WAL archiving to Cloudflare R2, verified by a restore drill; secrets live in Git as SealedSecrets; the site is reached through Cloudflare Tunnel with no ports open to the internet; and Prometheus, Loki and two outside monitors email me when anything breaks.
- Prompt-Aware Image Token Pruning for Multimodal LLMs
Trained a 1.3M-parameter scorer that distills where LLaVA-1.5-7B attends (its layer-8 attention), pruning the image tokens the model would ignore before they reach the frozen LLM. Keeping only a quarter of the 576 image patches, it nearly matches the unpruned model on TextVQA (36.0 vs. 36.7) and beats random pruning by 12.3 points, and its forward pass takes 0.43 ms per image, against 65 ms for a CLIP-GradCAM scorer. An alignment diagnostic across three VQA benchmarks showed why the training-free shortcut fails: what CLIP finds salient barely tracks what the LLM attends to. A Rice COMP 646 project with Chao Hsuan Ho and Kerstin Sun.
- Defending LLM Agents Against Tool Schema Poisoning
Reproduced a tool-schema poisoning attack in which deceptive parameter descriptions trick function-calling agents into filling in private data, establishing a 56% attack-success rate and a 52% privacy-leak score on Llama 3.1-8B. Built a representation-level defense: probing hidden states located a linear direction that separates leaking from safe tool calls (96% probe accuracy at layer 10, read at the end of the model's reply), and steering activations along it at inference cut the privacy-leak score by 14.5 points with no loss in original task success. An in-context reflection baseline cut leakage about as much but raised the attack rate and lowered task success, in both single- and multi-turn conversations; steering, tested single-turn, avoided both. A Rice COMP 584 team project, in which I built the probes and the steering.
- Cross-Size Generalization for GNN Job Shop Scheduling
Studied why GNN schedulers trained with reinforcement learning collapse on larger Job Shop instances: a GIN trained on 10×10 instances scores 136% above the best known schedules on Taillard 15×15. Tested the hypothesis that the cause is node degree in the disjunctive graph, which grows with the number of jobs. A 5-seed factorial showed that degree-normalized (mean) aggregation and machine-hub routing are two paths to the same ~26% error ceiling, letting a single policy trained on 10×10 instances generalize zero-shot to instances up to 20× larger, on par with the published L2D model. Built the research framework in plain PyTorch: GNN encoders (SAGE, GCN, GIN, GAT, PNA) and critic-free GRPO plus PPO trainers, benchmarked against dispatching rules on the FT, LA and TA sets. A Rice team project with Kerstin Sun and Cheng Han Lin.
- Agentic Hybrid Graph & Vector RAG System
Implemented a self-routing agentic AI system that dynamically chooses between Graph RAG (Neo4j), Vector RAG (Milvus), or a Hybrid path based on query intent. Developed a low-cost Graph RAG construction method using K-means and DBSCAN clustering on vector nodes, reducing indexing costs by 35% while maintaining performance.
- Multi-Agent AI System With RAG
This project leverages a multi-agent framework using CrewAI and RAG to orchestrate expert-level collaboration between LLMs, achieving a 90% success rate on complex algorithmic challenges. Recognized for its robust architecture and innovative implementation, the system secured an Outstanding Award, ranking in the Top 3 out of 50 teams at a premier Computer Science AI workshop.
- Relational Database Engine from Scratch
Built a relational database engine in C++, from the storage layer up to SQL. Storage: a buffer manager with LRU eviction and page pinning, slotted-page record storage, and a B+ tree index supporting range scans. Query processing: relational operators including a sort-merge join over an external merge sort that spills to disk, a SQL compiler, and a cost-based optimizer that orders joins from cardinality statistics, running the TPC-H benchmark at scale factor 1 (1 GB).
- Ricebook: Full-Stack Social Network
A social network built as a React 19 single-page app on Redux Toolkit, backed by an Express 5 and MongoDB API. Features session-cookie authentication with Google OAuth 2.0 account linking, a follower graph, a comment-threaded article feed, and Cloudinary image upload. The Redux store is covered by 30 Vitest suites at 97% statement coverage and the API by Jasmine integration tests; frontend and backend are deployed to separate hosts with CORS-aware session configuration.
- Chat Bar - Real-Time Multiplayer Game Server
Engineered a real-time multiplayer game server using TCP socket programming with multi-threaded client handling to support concurrent connections. Built a secure user management system featuring SHA-256 password hashing and optimized MySQL queries for ranking leaderboards. Developed an event-driven game engine for real-time state updates and group chatting.
- AI Fitness Motion Classification with MediaPipe
Built an end-to-end pipeline to classify fitness exercises (squats, bench presses, deadlifts) from video data. Extracted 33 temporal skeleton keypoints using MediaPipe and trained LSTM and Random Forest models to achieve high-accuracy motion recognition.
- Elderly Care Map Platform
Developed a full-stack interactive search platform for 2,000+ care institutions. Designed a normalized database schema with 8 tables and implemented an automated ETL pipeline to integrate government open data. Features include radius-based filtering, real-time distance calculation, and a CRUD-based user authentication system.
- Bayesian Optimization & Genetic Algorithm for Job Shop Scheduling
Solved the Job Shop Scheduling Problem with a genetic algorithm in Python and tuned its hyperparameters with Bayesian optimization. Automated exporting the resulting schedules into a FlexSim-compatible format, so every solution could be cross-validated in a discrete-event simulation.
- AI Game Agents Tetris Arena
A Tetris game built with PyGame, with human play, AI-assisted play, and AI-versus-AI competition modes. Implemented agents with a genetic algorithm, Q-learning and Deep Q-learning, then compared their capability, performance and complexity across different setups.