Do employer matching contributions in an RRSP count towards the contribution room?

Yes it does. The employer contribution is both a Taxable Benefit and also counts in the RRSP limit for that year. The Tax Receipt for RRSP Contributions you get from the financial institution where you have your RRSP will show the total amount deposited in your account, regardless of the source, employee or employer.

Excess contributions over 2000$ are usually taxed 1% per month : http://www.cra-arc.gc.ca/tx/ndvdls/tpcs/rrsp-reer/cntrbtng/xcss-eng.html

Yes:

Your contributions plus your employer's contributions cannot exceed your available RRSP room as reported on your notice of assessment for the previous year.

(Source: https://www.moneysense.ca/columns/group-rrsp-at-work/, emphasis added)

But why? When an employer makes an RRSP contribution on your behalf, they are in effect paying you more.

Example: You make $60,000 per year, and you are enrolled in an employer RRSP matching program that matches up to 5% of your annual income. You get 26 paycheques each year, and you've opted in to maximize the employer matched contributions. Your employer will deduct 5%, or $115.38, of your money from each paycheque and contribute that to your RRSP, along with a matching $115.38 of their money. That means your employer has actually payed you an extra $3,000 over the course of the year, it's just that the extra $3,000 was put into your RRSP instead of your pocket. But if they've payed you more, they should withhold more income from each paycheque for income tax, right?

Actually, it turns out that any money an employer has contributed to your RRSP on your behalf instead of paying you with gets deducted from the amount the employer has to withhold for income tax (source). That means you've already avoided paying income tax on that extra money. And that's also why these contributions count toward your RRSP contribution limit. If they didn't, you would be able to avoid paying taxes on it twice: once when your employer didn't include it in the amount they withheld from your paycheque for income tax, and twice when you claimed the equivalent amount as an RRSP contribution on your tax return form.