Which probability distribution will you use to model outliers?

I think the answer is Gaussian distribution. This is a famous approach that is used in Anomaly Detection. What you do is to fit your feature to the Gaussian distribution and the samples which have the probability below the specific threshold are labeled as an outlier.

Quoting from the paper Modeling Outlier Score Distributions:

Many existing unsupervised outlier detection algorithms calculate some kind of score per data object which serves as a measure of the degree of outlier. Scores are used in ranking data points such that the top n points are considered as outliers. For example, the statistical-based approach proposed in [4], uses a Gaussian mixture model to represent normal behavior and each datum is given a score on the basis of changes in the model. A high score indicates a high possibility of being an outlier.

Example of usage - 1.

Example of usage - 2