{"data":{"slug":"amitshekhariitbhu-ai-engineering-interview-questions","name":"ai-engineering-interview-questions","tagline":"Cheat Sheet for AI Engineering Interview","github_url":"https://github.com/amitshekhariitbhu/ai-engineering-interview-questions","owner":"amitshekhariitbhu","repo":"ai-engineering-interview-questions","owner_avatar_url":"https://avatars.githubusercontent.com/u/9877145?v=4","primary_language":"Markdown","stars":3112,"forks":564,"topics":["agents","ai","ai-agents","ai-engineering","fine-tuning","interview","interview-preparation","interview-questions","llm","llm-inference","mcp","quantization","questions-and-answers","rag"],"archived":false,"github_pushed_at":"2026-09-19T08:22:32+00:00","maintenance_label":"Very active","stars_delta_30d":860,"url":"https://www.graphcanon.com/tools/amitshekhariitbhu-ai-engineering-interview-questions","markdown_url":"https://www.graphcanon.com/tools/amitshekhariitbhu-ai-engineering-interview-questions.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/amitshekhariitbhu-ai-engineering-interview-questions","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=amitshekhariitbhu-ai-engineering-interview-questions","description":"Your Cheat Sheet for AI Engineering Interview – Questions and Answers.","homepage_url":"https://outcomeschool.com/program/ai-and-machine-learning","license":"Apache-2.0","open_issues":2,"watchers":41,"ai_summary":"A collection of questions and answers aimed at preparing candidates for AI engineering interviews, covering topics such as agents, fine-tuning, llm, quantization, and rag.","readme_excerpt":"### AI Infrastructure and Scalability\n\n- How do you improve inference speed in production LLM deployments?\n  - Answer: [LLM Inference Optimization](https://www.youtube.com/watch?v=jV2sCj4lHYk)\n- LLM optimization techniques\n  - Answer: [LLM optimization techniques](https://www.linkedin.com/posts/pallavi-shekhar_5-llm-optimization-techniques-lets-understand-activity-7442067281532325888-4aOS)\n- How do you select GPUs for LLM inference?\n  - Answer: [How does a GPU work for Deep Learning?](https://outcomeschool.com/blog/how-does-a-gpu-work-for-deep-learning)\n- How does a GPU work for Deep Learning?\n  - Answer: [How does a GPU work for Deep Learning?](https://outcomeschool.com/blog/how-does-a-gpu-work-for-deep-learning)\n- How does a Google TPU work?\n  - Answer: [How does a Google TPU work?](https://outcomeschool.com/blog/how-does-a-google-tpu-work)\n- How does an LPU work?\n  - Answer: [How does an LPU work?](https://outcomeschool.com/blog/how-does-an-lpu-work)\n- Estimate the GPU memory needed to serve a 70B model (weights, KV cache, activations). Does it fit on a single 80 GB GPU?\n- Do the roofline math: how many tokens/sec can one H100 produce for a 70B model at batch size 1?\n- What is model parallelism vs data parallelism in distributed training?\n- What is tensor parallelism, and how does it help serve large models?\n- What is pipeline parallelism?\n- How does continuous batching improve LLM inference throughput?\n  - Answer: [Continuous Batching in LLMs](https://outcomeschool.com/blog/continuous-batching-in-llms)\n- What is speculative decoding, and how does it speed up inference?\n  - Answer: [Speculative Decoding](https://outcomeschool.com/blog/speculative-decoding)\n- How does Medusa (multi-head speculative decoding) work?\n  - Answer: [Medusa - Multi-Head Speculative Decoding](https://outcomeschool.com/blog/decoding-medusa)\n- How does EAGLE (feature-level speculative decoding) work?\n  - Answer: [EAGLE - Feature-Level Speculative Decoding](https://outcomeschool.com/blog/decoding-eagle)\n- What is N-gram Speculation in LLMs, and how does it speed up generation?\n  - Answer: [N-gram Speculation in LLMs](https://outcomeschool.com/blog/n-gram-speculation-in-llms)\n- What is KV cache, and how do you manage memory for it?\n  - Answer: [What is KV Cache in LLMs?](https://outcomeschool.com/blog/kv-cache-in-llms)\n- What is Paged Attention?\n  - Answer: [Paged Attention in LLMs](https://outcomeschool.com/blog/paged-attention-in-llms)\n- How does GGUF work?\n  - Answer: [How does GGUF work?](https://outcomeschool.com/blog/how-does-gguf-work)\n- How do you optimize inference for edge and mobile deployment?\n  - Answer: [How does llama.cpp run LLMs on everyday hardware?](https://outcomeschool.com/blog/how-does-llama-cpp-run-llms-on-everyday-hardware) and [How does Model Quantization work?](https://outcomeschool.com/blog/how-does-model-quantization-work)\n- What is model quantization (INT8, INT4, FP16, BF16), and how does it affect quality?\n  - Answer: Explained in this video: [AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization](https://www.youtube.com/watch?v=lnfWvX66FUk) and [How does Model Quantization work?](https://outcomeschool.com/blog/how-does-model-quantization-work)\n- How do you implement auto-scaling for AI workloads?\n- What is the role of load balancing in AI serving infrastructure?\n- How do you manage GPU memory for serving multiple models?\n- What is model sharding, and when would you use it?\n- How do you implement request queuing and priority scheduling for AI services?\n- What are the cost trade-offs between self-hosted and API-based AI inference?\n- How do you handle cold start latency for serverless AI deployments?\n- How do you implement model caching to reduce redundant computations?\n  - Answer: [How does Prompt Caching work?](https://outcomeschool.com/blog/how-does-prompt-caching-work)\n- What is the difference between synchronous and asynchronous inference, and when do you use each?\n- What is FSDP (Fully Sharded Data P","github_created_at":"2026-03-21T07:28:38+00:00","created_at":"2026-07-11T11:38:21.465475+00:00","updated_at":"2026-09-20T06:28:59.379299+00:00","categories":[{"slug":"ai-agents","name":"AI Agents","url":"https://www.graphcanon.com/categories/ai-agents","markdown_url":"https://www.graphcanon.com/categories/ai-agents.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/ai-agents"},{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"agents","name":"agents"},{"slug":"ai-engineering","name":"ai-engineering"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"llm","name":"llm"},{"slug":"quantization","name":"quantization"}],"trust":{"provenance":{"is_fork":false,"github_id":1187805808,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-19T18:23:42.895Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":0,"last_release_at":null,"stars_delta_30d":860,"open_issues_delta_30d":1},"security_summary":{"status":"no_manifest","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:38:22.718Z","medium_count":0,"scan_profile":"mcp_manifest","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-19T18:23:43.891Z"},"languages":{"value":["markdown"],"source":"github.language","observed_at":"2026-09-19T18:23:43.891Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-19T18:23:43.891Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When looking to prepare for specific AI engineering interview topics such as agents or model fine-tuning","For individuals focusing on practical aspects of large language models (llm) that require an understanding beyond basic concepts"],"when_not_to_use":["If the preparation focus is solely on theoretical knowledge without practical question scenarios","When aiming to prepare for a more general software engineering position not specifically centered around AI technology or its implementation details"],"source":"enrich:decision_facts","observed_at":"2026-07-15T10:20:07.943Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"A collection of questions and answers for preparing candidates specifically for AI engineering interviews, with notable inclusions on agents, fine-tuning, llm, quantization, and rag."}]}}