ARTICLE AD BOX
I’m building a macOS desktop app (C#/.NET + Avalonia) that:
Captures a small region of the screen and checks pixel colors (detect a green/red “signal panel”).
If a signal is detected, it sends a mouse click to a configured Buy/Sell screen point using CoreGraphics events.
What works
Mouse clicks land correctly when I post a CGEvent mouse click. This works without any Y-axis inversion (which makes me think my “click coordinate system” is already correct). The app has Accessibility permission (to control the mouse) and Screen Recording permission (to capture the screen).What fails
My capture sometimes returns an image of the desktop wallpaper / background, as if it’s capturing what’s behind the visible TradingView window (Chrome), rather than the pixels the user can see.
When this happens:
My pixel sampling always reports “no signal” because it’s sampling wallpaper colors instead of the signal panel. It occurs even while Chrome/TradingView is clearly frontmost and visible on screen.Why I think this is not just a simple Y-flip issue
If it were only a coordinate flip/offset problem, I’d expect the capture to show a wrong part of the TradingView window, not the desktop behind it. That makes me wonder if this is caused by:
macOS permission behavior, window compositing / GPU-accelerated rendering in Chrome, or using the wrong API for “capture what the user sees”.This is the capture function. The rect is intended to be global screen coordinates of the region where the signal panel is drawn.
// Rect is intended to be in global screen coordinates var rect = new CGRect(x, y, width, height); // Capture an on-screen image for that rect using var img = CGWindowList.CreateImage( rect, CGWindowListOption.OnScreenOnly, CGWindowID.Null, CGWindowImageOption.Default ); // Save debug image to confirm what was captured if (img == null) { Console.WriteLine("Capture returned null"); } else { SaveToPng(img, "debug.png"); }Questions
Under what conditions will CGWindowListCreateImage / CGWindowList.CreateImage(...) return the desktop/background behind visible windows even with OnScreenOnly?
Is this a known limitation with Chrome / GPU-accelerated content (TradingView), where CoreGraphics capture doesn’t return the composited pixels?
What is the recommended modern approach for capturing the actual visible screen pixels on macOS (e.g., should I use ScreenCaptureKit instead)?
If CGWindowListCreateImage can work here, what flags/options or coordinate conversions are required to make it reliable?
Environment: Apple Silicon Mac, single display (can provide resolution + multi-monitor details if needed).
