Dan Peng

Deployed an agentic AI reporting system for clinical operations on GCP, taking a run from about 3 hours to under 2 minutes.

• LangGraph orchestration with async execution, shared state, retrieval and self-reflection loops, and human-in-the-loop review: end-to-end runs from about 3 hours to under 2 minutes.
• Hybrid RAG retrieval: BM25 plus embedding search in Elasticsearch, with LLM-generated chunk summaries, took Recall@5 from 75.1% to 91.2% on a 10K-query evaluation set.
• Node- and tool-level tracing and evaluation in LangSmith across 100K simulated queries, tracking latency, tokens, cost and answer quality per agent trajectory, cut inference cost by about 30% with no drop in quality.
• Inside the vLLM inference engine, KV-cache management and scheduler improvements cut P95 latency 3× while sustaining 5.3K tokens a second on production workloads.
• A reproducible MLOps pipeline on W&B, Docker and GCP for dataset and model versioning, experiment tracking, automated evaluation gates and production rollback.

First author of SycoEval-EM, an open-source multi-agent benchmark of how LLMs hold up when patients push back in emergency care.

• A containerized benchmark with patient, physician and evaluator agents over MCP, run across 19 LLMs and 1,425 encounters. Failure-log analysis and prompt engineering raised tool-calling success by 20%.
• An LLM-as-judge pipeline that agrees with blinded ratings from two physicians at Cohen's κ = 0.96, and keeps 99.4% leave-one-judge-out concordance in sensitivity analysis.
• Mixed-effects regression showed model identity explained 64% of the variance in safety risk, which informs guardrails and pre-deployment stress tests. Under review at npj Digital Medicine.

Built PROwl, an event-driven AI code-review agent, and rolled it out across 3 engineering teams.

• GitHub webhooks trigger the agent on AWS EKS (Kubernetes), which took median time to first review from 6 hours to under 10 minutes.
• A hybrid knowledge-graph and embedding index, built with LlamaIndex entity and relation extraction and stored in PostgreSQL, over 10 API manuals and 5 repos in S3, grounds feedback in client-specific context.
• SQL analysis of reviewer accept and reject signals on PR events drove prompt iteration, raising accepted agent comments from 27% to 55%.
• CI/CD on GitHub Actions with automated test and 97% coverage gates before every agent release.
• Led weekly technical reviews with 8 engineers to gather workflow and integration requirements, using adoption and efficiency metrics to guide the wider rollout.

Multimodal clinical trial prediction, high-risk order detection, and faster model training.

• A multimodal mixture-of-experts transformer in PyTorch Lightning for clinical trial outcome prediction, fusing text and drug formulations, state of the art on phases I, II and III. Co-author at EMNLP 2025 Findings.
• Gradient-boosted (XGBoost) anomaly prediction on 10+ engineered feature tables in BigQuery: AUC 0.87, 0.21 above an LSTM baseline, with SHAP reason codes that flag high-risk clinical orders.
• Multi-GPU training with PyTorch DDP and mixed precision, from 10 hours to 2.