TransWikia.com

How do I use python to get the UI elements of a native mac app?

Stack Overflow Asked on December 25, 2021

I need to get the text from UI elements on a mac GUI.

One possible route might through pyobjc and UIAccessibility?

https://developer.apple.com/documentation/objectivec/nsobject/uiaccessibility
https://pyobjc.readthedocs.io/en/latest/core/intro.html

I’ve RTFM for hours and can’t figure it out. It doesn’t help that I’m not a pyobjc or mac developer. I assume it’s something so easy, there’s no example for it online that I can find.

Maybe we can add a good example as the answer to this question 🙂

2 Answers

There is some information you can access about the windows and titles using the Python Quartz package, but it sounds like you want items rendered in the window. If it's just window info, use Quartz, here's the gist:

import Quartz
windows = Quartz.CGWindowListCopyWindowInfo( Quartz.kCGWindowListOptionAll, Quartz.kCGNullWindowID)
print windows[0].valueForKey_(kCGWindowBounds)
...

If you want the items rendered in the window, you'll need to be able to modify the child program so it sets NSWindow.sharingType = .readOnly. In addition, you'll most likely need to have spawned the child process. After all that, you may be able obtain a handle to the NSWindow using the CGWindowID:

 import AppKit
 windowID = windows[0].valueForKey_(kCGWindowNumber)
 window = NSWindow.windowWithWindowNumber(windowID)
 print window.contentView.subViews[0]
 ...

and introspect the NSWindow.contentView to find your target subview of type NSTextField, etc.

I haven't tried the second part using NSWindow.windowWithWindowNumber, but I have used the Quartz package in other projects successfully.

Answered by Marc Lavergne on December 25, 2021

From what I remember from previous tries, pyatomac proved pretty useful. They have a quite good documentation.

Answered by wschopohl on December 25, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP