Eye-AF from Nikon Z6 is inconsistent or fidgety

I have a Nikon Z6 – latest firmware (ver. 3.12, from October 2020). As far as I can remember since Eye-AF was first introduced in May 2019, I’ve been having a hard time have the eye-AF locked on the subject.

My camera’s autofocus would basically go haywire trying to lock it in. I don’t know what is wrong, or maybe I’m misusing it.

My expectation: eye-AF would lock onto an eye. If I wanted to, I would be able to press left or right on the panel and it would lock onto another eye.

What it is currently doing: it would fidget/jump around and tries to get a lock. I am half-clicking to maintain focus, but it would bounce around so bad that you can hear the mechanism "seek" the focus.

It does this regardless of which lens is attached, or the aperture settign. I have 35mm, 50mm, and 85mm F/1.8 lenses, and it does it with all of them. I am still able to capture great pictures; however this "fidget" thing is bothering me as obviously I need several shots to get what I need in focus.

It does this in the field and in the studio.

Any help, indicators, experience to resolve this would be greatly appreciated.

UPDATE

Per comments:

• subject is stationary (still). AF square rotates between eye to eye and to face (larger square)
• Focus Mode: AF-C setting
• WB: auto
• AF-area mode: Auto-area AF
• release mode: single frame
• metering: matrix
• active D-lighting: off
• vibration reduction: on