Getting an sf boundary of a raster or stars object

I am trying to figure out the best approach to convert an raster or stars object into its boundary returned as an sf object. I think I have a pretty decent handle on how to do it with raster (shown below):

library(sf)
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
library(stars)
#> Loading required package: abind
library(raster)
#> Loading required package: sp

## path
test <- system.file("external/test.grd", package = "raster")


## Using raster
x <- raster(test)
r <- x > -Inf
pp <- rasterToPolygons(r, dissolve = TRUE)
#> Loading required namespace: rgeos
boundary_x <- st_as_sf(pp)
plot(boundary_x)

Here using stars i am forced to rely on st_union for that final aggregation step. For this example, it is pretty quick but for larger examples I first have to create many little polygons (using st_as_sf.stars then use st_union) which takes a long time. Is there a more efficient way of doing this?

## Using stars
y <- read_stars(test)
boundary_y <- st_as_sf(y, merge = TRUE) %>%
  st_union()
plot(boundary_y)