Custom LLM applications built on your data with EdgeFirm

Common Questions About Custom LLMs

What's the difference between RAG and fine-tuning?

RAG (Retrieval Augmented Generation) is best for factual Q&A where you need sourced, up-to-date answers. It searches your documents and generates answers with citations. Fine-tuning is best when you need consistent style/format or specialized terminology. It trains the model on your examples. Most successful systems use both: fine-tune for your domain and style, RAG for factual grounding. RAG is usually cheaper, faster to deploy, and easier to update. Fine-tuning usually costs more but works offline and delivers more consistent behavior.

How accurate can you make the AI?

It depends on your data and the task, so we measure it on your own documents. We build an evaluation set with known answers, add confidence scoring so the system says when it is unsure, send low-confidence cases to a person, and monitor quality after launch. We share the test method and results with you. A fixed accuracy figure without a test set means little: Stanford researchers found that legal research tools sold as reliable were wrong 17% to 33% of the time. If your use case needs accuracy that current technology cannot deliver, we will tell you.

What if my data is messy or incomplete?

Very common. Many projects start with data quality issues. Our approach: Phase 1 (Data Audit) assesses completeness, quality, structure, and estimates cleanup effort. Phase 2 (Data Preparation) does automated cleaning and manual curation for critical data. Phase 3 (Continuous Improvement) monitors answer quality and identifies gaps from usage. Sometimes we recommend a 1-2 month data prep phase before building AI. We can also start with your best data and expand coverage over time.

Can you integrate with our existing systems?

Yes, we specialize in integration. Common integrations: Document repos (Confluence, SharePoint, Google Drive, Notion), Communication (Slack, Teams, email), Databases (SQL, NoSQL, APIs), CRM (Salesforce, HubSpot, Zendesk), File storage (AWS S3, Box, Dropbox), Authentication (SSO via Okta, Azure AD, Google). If you have APIs, we can integrate. If you don't, we can build scheduled exports, webhook listeners, custom connectors, or direct database access.

How do you handle sensitive data?

Security and privacy by design. Data Protection: Encryption at rest (AES-256) and in transit (TLS 1.3), access controls, PII detection and masking, audit logs. Deployment Options: Your cloud account (data never leaves your tenant), on-premise (never leaves your network), VPC isolated, or air-gapped. Compliance: HIPAA BAA available, GDPR data locality and deletion, SOC 2 controls. Model Privacy: Azure OpenAI stays in your tenant, self-hosted models give complete control.

What is RAG as a service?

We run and maintain your RAG system for you. That means keeping indexes current, adding new sources, and reviewing answer quality each month. It suits teams that want the results without hiring people to run vector databases and model operations.

What happens when documents are updated?

Automatic or scheduled updates. Real-time: Webhook triggers when documents change, auto-reindex with 5-15 minute lag. Scheduled: Daily/weekly/monthly re-indexing, incremental updates (only changed docs). Version Control: Maintain document history, query specific versions, track when information changed. Most systems use a combination: real-time for critical docs, nightly batch for everything else.

Can we use our own models or APIs?

Yes. We design for flexibility. Model options: OpenAI (GPT-4, GPT-3.5), Anthropic (Claude 3.5), Azure OpenAI (in your tenant), AWS Bedrock (Claude, Llama, Mistral in your AWS), or self-hosted open-source (Llama 3.1, Mistral, Mixtral). We build an abstraction layer so you're not locked into one provider. You can start with OpenAI and switch to Claude later, A/B test models, or route based on query complexity.

How much does a custom LLM application cost?

Projects start from $20K at a fixed price, and a full build is delivered in 4 to 5 months. The main drivers are data complexity (100 documents versus 1 million), how sophisticated the use case is (simple Q&A versus agent workflows), how many users and queries it needs to support, and your deployment and compliance requirements. We model the number with you before you sign. A full build includes integration with 3-5 systems, the interface, admin dashboard, documentation, and 30 days of post-launch support.

What happens if accuracy drops after launch?

We build confidence scoring into every system, so answers the model is unsure about go to a person instead of out the door. After launch we monitor quality against the evaluation set we built during discovery, and if accuracy drifts (new documents, changed terminology, model updates upstream) we can trace it back to a cause. The 30 days of post-launch support covers this tuning. After that, most clients either maintain it themselves with the documentation we hand over, or move to a retainer we agree individually based on actual support needs.

Do we need an in-house AI or ML team to run this afterward?

No. We hand over the code, infrastructure, and documentation, and train your team on day-to-day management (adding documents, reviewing flagged answers, reading the monitoring dashboard). You don't need data scientists on staff to keep it running. If you want ongoing development, model upgrades, or new data sources added without hiring, that is what RAG as a Service and retainer support are for.

How is this different from an off-the-shelf AI tool like ChatGPT Enterprise or Glean?

Those platforms are good general-purpose tools, but they are built for the average company, not yours. We build on your exact data, your exact workflows, and we test accuracy on your own questions, not a generic benchmark. You own the code and infrastructure instead of paying per-user licensing forever, and we can integrate directly into your CRM, ticketing, or internal tools rather than leaving you inside a separate chat window.