Abstract
Foundation models contain a wealth of information from their vast number oftraining samples. However, most prior arts fail to extract this information ina precise and efficient way for small sample sizes. In this work, we propose aframework utilizing reinforcement learning as a control for foundation models,allowing for the granular generation of small, focused synthetic support setsto augment the performance of neural network models on real data classificationtasks. We first allow a reinforcement learning agent access to a novel contextbased dictionary; the agent then uses this dictionary with a novel promptstructure to form and optimize prompts as inputs to generative models,receiving feedback based on a reward function combining the change invalidation accuracy and entropy. A support set is formed this way over severalexploration steps. Our framework produced excellent results, increasingclassification accuracy by significant margins for no additional labelling ordata cost.