Design an LLM Assistant Deployed in a Customer’s Cloud · System Design Interview

Design an LLM Assistant Deployed in a Customer’s CloudAs a forward deployed engineer, design an assistant that answers employees’ questions from their company’s internal documents and tickets, and that runs inside the customer’s own cloud account.What it has to do

Answer a natural-language question, with citations to the internal sources it used

Use only sources the asking employee is allowed to read

Pick up new and changed content from the customer’s existing systems

Scale

Employees: 40,000

Documents across connected systems: 30 million

Out of scope

Training or fine-tuning the model

The chat interface

Taking actions in other systems

Billing and licensing

Practise this challengeAll system design challenges

LetsDiscuss

Loading your session…