Say you are a large, but non-tech, company like a bank looking at investing resources in genAI. Most ‘normal’ companies aren’t going to be training new base models. But what does the escalating spectrum of potential investment look like these days: prompt engineering > refining / custom GPT > RAG > ?? > Training your own (small) transformer model

@Tom Purves This is sufficiently broad that it likely needs some discussion: It will depend on your use cases and requirements for those use cases, as in who and how they will be used. 1. (Retrieval/Searching) Suppose you are talking about validated information retrieval from your databases. In that case, RAG, will be essential, and it has the potential to be evaluated for accuracy before deployment, which is essential for regulated markets. 2. (Generating) If you are talking about generating documents with similar styles as other documents, implicit representations in finetuning can help, but often enough, providing style guidance examples can work. This can increase cost overhead because of the added tokens, making some light finetuning not unreasonable to consider, as it will also help make inference a bit faster. --> This will depend on the requirements. You may have different levels of requirements for internal use vs External uses. Many times, because of the specificities of these different uses, it may entail partnering with companies focusing on specific areas that can enable whole-chain solutions (prompts, chains, finetuning, etc., and have the benefit of having costs distributed at scale. Legal is a great example. Generally, knowing what is 'good enough' and how you intend to measure it will help to guide the process more precisely than your good starting point of

prompt engineering > refining / custom GPT > RAG > ?? > Training your own (small) transformer model