Add class to {{ link }} twig attribute

This is how to do it:

{{ link(item.title, item.url, { 'class': ['text-blue500', 'text-3xl'] }) }}

You will have to rebuild the link(a href tag) with relevant attributes you require. You can add class, id, download or target blank as well. Refer definition below -

function THEME_preprocess_file_link(&$variables)
{

  $file = $variables['file'];
  $link_text = '';
  $options = array();
  $file_entity = $file instanceof File ? $file : File::load($file->fid);
  $url = file_create_url($file_entity
    ->getFileUri());
   // Use the description as the link text if available.
  if (empty($variables['description'])) {
    $link_text = $file_entity
      ->getFilename();
  }
  else {
    $link_text = $variables['description'];
    $options['attributes']['title'] = $file_entity
      ->getFilename();
  }

  $variables['attributes']->addClass(array('cta'));//Adding class to span
  $options['attributes']['download'] = TRUE;//Adding attribute to l
  $options['attributes']['target'] = array('_blank');//Adding attribute to l
  $variables['link'] = Drupal::l($link_text, Url::fromUri($url, $options));

}

I think your easiest approach would be to modify the field template and add a wrapping span with classes you need.

field--field-name.html.twig:

{% for item in items %}
  <span{{ item.attributes.addClass('my-class') }}>{{ item.content }}</span>
{% endfor %}

or replace span and content with to add class directly to <a> tag:

<a href="{{ item.content['#url']}}" class="my-class">{{ item.content['#title'] }}</a>