Wrapping fields together in a Views Exposed Filter

I’m trying to insert divs (or any tag) in my exposed filter form with the purpose of grouping related fields together. I want to have something like this:

<div class="my-group">
  field one
  field two
  ...
</div>

So far, I have tried this in my hook_form_alter(), with no luck, since the prefix and sufix are added within field_one, instead of before field one and after field two:

$form['field_one']['#prefix'] = '<div class="my-group">';
$form['field_two']['#suffix'] = '</div>';

How can I accomplish this (without external modules)?

  // Create a fieldset to group related fields.
  $form['my_group'] = array(
    '#type' => 'fieldset',
    '#attributes' => array(
      'class' => array('my-group')
    ),
  );

  // Move the desired fields in it.
  $form['my_group']['field_one'] = $form['field_one'];
  $form['my_group']['field_two'] = $form['field_two'];

  // Unset the original fields, so they don't show up in the form anymore.
  unset($form['field_one']);
  unset($form['field_two']);