What is the opposite of "brute-force"?

I believe that you are speaking of optimization algorithms or heuristic algorithms.

There is a great variety of these techniques. For example:

A heuristic algorithm is one that is designed to solve a problem in a faster and more efficient fashion than traditional methods by sacrificing optimality, accuracy, precision, or completeness for speed. In these problems, there is no known efficient way to find a solution quickly and accurately although solutions can be verified when given. Heuristics can produce a solution individually or be used to provide a good baseline and are supplemented with optimization algorithms. Heuristic algorithms are most often employed when approximate solutions are sufficient and exact solutions are necessarily computationally expensive

Northwest university

The right choice of an optimization algorithm can be crucially important in finding the right solutions for a given optimization problem. There exist a diverse range of algorithms for optimization, including gradient-based algorithms, derivative-free algorithms and metaheuristics. Modern metaheuristic algorithms are often nature-inspired, and they are suitable for global optimization. In this chapter, we will briefly introduce optimization algorithms such as hill-climbing, trust-region method, simulated annealing, differential evolution, particle swarm optimization, harmony search, firefly algorithm and cuckoo search

Researchgate