Improve an LLM's reasoning on a domain task

A model reasons poorly on one specialized task. How would you improve it? Compare the options, from prompting and retrieval to fine-tuning and reinforcement learning, say what data each one needs, and explain how you would measure that reasoning on the task actually got better.