The app doesn't stop after failing golden test. + Error in TestGoldenComparatorProcess #174267
Description
Steps to reproduce
- Clone the repo. It has a sample golden test with an image that fails the test intentionally.
- Launch iOS simulator
- Run in the terminal
flutter test integration_test/E2Es
Expected results
The test process stops. I should not kill it manually.
Actual results
The test process doesn't exit having the following output:
<< [ERROR:flutter/shell/testing/tester_main.cc(682)] Unhandled exception
<<< Exception: Golden "goldens/failing_counter.png": Pixel test failed, 0.11%, 4496px diff detected.
<<< Failure feedback can be found at /Users/x/Documents/Flutter/golden_tests/integration_test/E2Es/failures
<<< Stack trace: #0 LocalFileComparator.compare (package:flutter_test/src/_goldens_io.dart:108:5)
<<< <asynchronous suspension>
<<< #1 main (file:///var/folders/6n/2pnrqh45205_wq6kvgs8xmq00000gn/T/flutter_tools.40hhzs/flutter_web_platform.TmZB1C/listenerHyn8yD/listener.dart:29:26)
<<< <asynchronous suspension>
<<<
00:23 +0 -1: loading /Users/x/Documents/Flutter/golden_tests/integration_test/E2Es/ui_test.dart [E]
type 'Null' is not a subtype of type 'Map<String, dynamic>'
dart:async/stream_impl.dart 1038:17 _StreamIterator.current
package:flutter_tools/src/test/test_golden_comparator.dart 315:27 TestGoldenComparatorProcess.getResponse
===== asynchronous gap ===========================
package:flutter_tools/src/test/test_golden_comparator.dart 161:41 TestGoldenComparator._compareGoldens
===== asynchronous gap ===========================
package:flutter_tools/src/test/test_golden_comparator.dart 127:28 TestGoldenComparator.compare
===== asynchronous gap ===========================
package:flutter_tools/src/test/flutter_platform.dart 554:17 FlutterPlatform._listenToVmServiceForGoldens.<fn>
Code sample
Code sample
import 'package:flutter_test/flutter_test.dart';
import 'package:golden_tests/main.dart';
import 'package:integration_test/integration_test.dart';
const kGoldensFolder = 'goldens';
void main() {
testWidgets('Counter smoke test', (WidgetTester tester) async {
await tester.pumpWidget(const MyApp());
expect(find.text('0'), findsOneWidget);
expect(find.text('1'), findsNothing);
await tester.pumpAndSettle(const Duration(seconds: 5));
await compareScreenshot(kGoldensFolder, 'failing_counter');
});
}
Future<void> compareScreenshot(String goldensFolder, String key) async {
List<int> screenshotBytes =
await IntegrationTestWidgetsFlutterBinding.ensureInitialized()
.takeScreenshot(key);
await expectLater(
screenshotBytes,
matchesGoldenFile('$goldensFolder/$key.png'),
);
}Screenshots or Video
Screenshots
Logs
No response
Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.35.1, on macOS 15.6 24G84 darwin-arm64, locale en-RU) [334ms]
• Flutter version 3.35.1 on channel stable at /Users/x/fvm/versions/3.35.1
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 20f8274939 (8 days ago), 2025-08-14 10:53:09 -0700
• Engine revision 1e9a811bf8
• Dart version 3.9.0
• DevTools version 2.48.0
• Feature flags: enable-web, enable-linux-desktop, enable-macos-desktop, enable-windows-desktop, enable-android, enable-ios, cli-animations, enable-lldb-debugging
[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.0) [1,087ms]
• Android SDK at /Users/x/Library/Android/sdk/
• Emulator version 35.2.10.0 (build_id 12414864) (CL:N/A)
• Platform android-35, build-tools 35.0.0
• ANDROID_HOME = /Users/x/Library/Android/sdk/
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
This is the JDK bundled with the latest Android Studio installation on this machine.
To manually set the JDK path, use: `flutter config --jdk-dir="path/to/jdk"`.
• Java version OpenJDK Runtime Environment (build 21.0.3+-79915917-b509.11)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 16.4) [685ms]
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 16F6
• CocoaPods version 1.16.2
[✓] Chrome - develop for the web [103ms]
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2024.2) [103ms]
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 21.0.3+-79915917-b509.11)
[✓] IntelliJ IDEA Ultimate Edition (version 2025.1.3) [102ms]
• IntelliJ at /Applications/IntelliJ IDEA.app
• Flutter plugin version 87.0
• Dart plugin version 251.27623.5
[✓] IntelliJ IDEA Community Edition (version 2024.3.6) [23ms]
• IntelliJ at /Applications/IntelliJ IDEA CE.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] VS Code (version 1.102.2) [6ms]
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension can be installed from:
🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[✓] Connected device (3 available) [5.7s]
• iPad Pro 11-inch (M4) (mobile) • 19A2CC4F-464F-4482-87E2-0B29AEED90A4 • ios • com.apple.CoreSimulator.SimRuntime.iOS-18-6 (simulator)
• macOS (desktop) • macos • darwin-arm64 • macOS 15.6 24G84 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 139.0.7258.139
[✓] Network resources [682ms]
• All expected network resources are available.
• No issues found!