The concept of “synthetic data,” or artificially made information, has recently created a stir. In today’s world, data is a tremendous asset for firms, and information can frequently provide a decisive competitive advantage. The idea of easily acquiring free data has spawned exaggerated claims and controversy.
However, as your mother most likely taught you, if anything appears too good to be true, it probably is.
However, manufactured data makes the reality more complicated. While we can’t stop gathering data and “just ask the model,” there are several exciting middle-ground applications for AI-generated data. And making good use of this data might help propel your firm forward. In this case, there is no free lunch, but there is the prospect of a complimentary side or two.
To help you better appreciate the opportunities that synthetic data presents, I’ll go over three basic strategies for generating fresh data. These are not the only options available, but they are the most commonly used nowadays.
- Direct Querying.
The first mode is the one most people associate with synthetic data: direct querying. When you first utilized ChatGPT or any other AI chatbot, you undoubtedly said to yourself, “Wait a second. Interviewing a Gen Z participant who is passionate about RPGs can be done similarly to interviewing a research respondent.
Working with this type of data might quickly become difficult or ineffective due to the age of the training datasets. Responses can be biased or contain unsuitable viewpoints, which can readily emerge. Furthermore, a big portion of the training data for these models is sourced from platforms such as Reddit, which can contain more controversial content than you’d prefer in your own dataset.
Aside from the red flags, the major issue with this type of data is that it is uninteresting. By definition, it generates plausible responses based on the sum of all its training. As a result, it tends to produce apparent solutions, which is the polar opposite of the type of insight we are typically seeking for. While direct questioning of the LLMs can be interesting, this method of producing massive amounts of fake data is unlikely to be the best approach.
2. Data augmentation.
We can progress beyond data searching by utilizing models to extract data from data that you provide them with — a process known as data augmentation. This strategy makes use of the reasoning and summarizing abilities of LLMs. Still, rather than relying exclusively on the original training data, you use models to analyze your own data and generate perturbations as if they were original data.
The procedure looks something like this. First, you must understand the information you are bringing to the table. Perhaps the data comes from an internal system, primary research, a trusted third-party supplier, segmentation, or the appending of desirable behaviors. After determining the source of your data, you may utilize the LLM to assess and submit additional data with comparable features.
This technique is significantly more promising, and it gives you power that LLMs alone cannot provide.
Many in the martech business may be thinking, “Like look-alikes?” You are accurate. The new models enable us to create look-alikes in ways that were previously impossible. This allows us to enhance or generate data that is consistent and comparable to the data we already have.
Having this much data is often useful for testing systems or examining some of the edges that a system may need to handle. It might also be used to deliver completely anonymous information for demonstrations or presentations. Avoid the cyclical thinking of “Let’s generate a ton of data and analyze it,” when it’s better to focus on the root data.
- Data retraining.
Finally, the final method of generating synthetic data is to keep a model that represents the data we have directly. The “holy grail” approach of taking a model and performing specific fine-tuning on a data set has been around for a long time, but until recently, it simply required too many resources and was much too expensive to be a viable option for most.
However, technology change. The prevalence of smaller but high-performance models (i.e., LLaMA, Orca, and Mistral), combined with recent revolutionary approaches to fine-tuning (i.e., Parameter Efficient Fine Tuning, or PEFT, and the LoRa, QLoRa, and DoRa sisters), allows us to effectively and efficiently produce highly customized models trained on our data. These are likely to be the strategies that genuinely make synthetic data shine, at least in the near future.
While there is no free lunch, and the risks of bias, boredom, and circular thinking are real, the benefits of synthetic data make it very appealing. And, when used effectively, it can result in efficiency and exponential potential.