Assign raster values based on polygon intersection

I have a raster where every cell is a 1. I also have a polygon of buffer areas that fall within the raster. I want to assign all raster cells that intersect the polygon geometrics to NA. But the two files have different extents and origins. I don’t see a reason to share code, since I build the raster and polygons based on proprietary data. But I can illustrate it. The black grid represents my current raster, the blue cicle is the polygon:

What function will allow me to reassign the raster value of all cells that intersect the polygon? The raster would become as follows:

library(raster)
library(sf)
m = matrix(rep(1,100), ncol = 10)
r = raster(m)

point = st_sfc(st_point(c(.8,.1))) # let's create a point and buffer it to have a polygon
pol = st_buffer(point,.1)
plot(pol, add = T)
r_masked = mask(r, as(pol, "Spatial"), inverse = T )

par(mfrow = c(1,2))
plot(r, axes = F, legend =F, bty = "n")
text(r)
plot(r_masked, axes = F, legend = F)
text(r_masked)
plot(pol, add = T)