Browser asks for password several times .htaccess using subdomain

I have a private homepage and I want to password-protect it using .htaccess. This is what my .htaccess file looks like:

AuthName MySite
AuthType Basic
AuthUserFile /what/ever/.htpasswd
require valid-user

#making embeded videos work in wordpress
<Files ~ ".(mp4|m4v)">
AddType video/mp4 .mp4 .m4v
order allow,deny
allow from all
satisfy any
</Files>

# Code for WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /whatever/
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /whatever/index.php [L]
</IfModule>
#End WordPress

Options -Indexes

It’s located in the folder directory directed for the homepage.

Problem is that when accessing the server via it’s subdomain the browser keeps asking for login/pass. If pressing cancel after two atempts one is directed to the page anyway.

If trying to access the webpage without subdomain, like this: www.mypage.com/privatesite, the browser asks for password once and then it’s done. No problems.

This is the homepages .conf file in apache:

<VirtualHost *:80>

ServerName my.site.com
ServerAlias www.my.site.com
ServerAdmin [email protected]
DocumentRoot /var/www/html/site
AccessFileName .htaccess

<Directory "/var/www/html/site">
AllowOverride All
Order allow,deny
Allow from all
Require all granted
</Directory>

         ErrorLog ${APACHE_LOG_DIR}/error.log
         CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>