We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
ChatGPT from Scratch: How to Train an Enterprise AI Assistant • Phil Winder • GOTO 2023
Train an enterprise AI assistant from scratch using ChatGPT, navigating quantization, transformers, LLMs, fine-tuning, reward models, adversarial attacks, data drift, prompt engineering, and more, for a holistic approach to natural language processing.
- Quantization: Reduce the number of bits to store or transmit a model, which can greatly reduce storage and transfer requirements.
- Transformers: A popular library used to support Hugging Face, which supports a range of natural language processing tasks.
- LLMs: Large Language Models, which are trained on vast amounts of text data and can generate text in response to prompts.
- Fine-tuning: The process of adapting a pre-trained LLM to a specific task by adding a small amount of custom data and re-training.
- Reward models: Models that are trained to evaluate the quality of generated text and provide feedback to the LLM.
- Adversarial attacks: Techniques used to test the robustness of LLMs by injecting malicious prompts or data into the model.
- Data drift: The phenomenon where the underlying distribution of the data changes over time, making it difficult for LLMs to generalize.
- Prompt engineering: The practice of crafting specific prompts to elicit desired responses from LLMs.
- Parameter efficient fine-tuning: A technique that uses the weights of a pre-trained model to adapt to a new task, reducing the amount of computation required.
- Holistic approach: A comprehensive approach to LLM development that considers numerous factors, including data quality, model architecture, and evaluation metrics.