Great crowd with the Paris Android User Group at Radio France
Showing how frame metrocs & percentiles are computed in Macrobenchmark, then I went on to explain how to draw a distribution, compute confidence intervals, and do bootstrapping. That was a little ambitious, but tons of fun
#AndroidFun
Showing how frame metrocs & percentiles are computed in Macrobenchmark, then I went on to explain how to draw a distribution, compute confidence intervals, and do bootstrapping. That was a little ambitious, but tons of fun
#AndroidFun
October 21, 2025 at 10:51 PM
Great crowd with the Paris Android User Group at Radio France
Showing how frame metrocs & percentiles are computed in Macrobenchmark, then I went on to explain how to draw a distribution, compute confidence intervals, and do bootstrapping. That was a little ambitious, but tons of fun
#AndroidFun
Showing how frame metrocs & percentiles are computed in Macrobenchmark, then I went on to explain how to draw a distribution, compute confidence intervals, and do bootstrapping. That was a little ambitious, but tons of fun
#AndroidFun
[French]
Mardi 21 Octobre (dans 2 semaines!) à 19h, le Paris Android User Group organise un meetup chez Radio France. Il paraît que le lieu est magnifique!
Je serai là en 2e partie pour vous parler de benchmarks dignes de confiance.
www.meetup.com/android-pari...
#AndroidDev #paris
Mardi 21 Octobre (dans 2 semaines!) à 19h, le Paris Android User Group organise un meetup chez Radio France. Il paraît que le lieu est magnifique!
Je serai là en 2e partie pour vous parler de benchmarks dignes de confiance.
www.meetup.com/android-pari...
#AndroidDev #paris
October 7, 2025 at 8:59 AM
[French]
Mardi 21 Octobre (dans 2 semaines!) à 19h, le Paris Android User Group organise un meetup chez Radio France. Il paraît que le lieu est magnifique!
Je serai là en 2e partie pour vous parler de benchmarks dignes de confiance.
www.meetup.com/android-pari...
#AndroidDev #paris
Mardi 21 Octobre (dans 2 semaines!) à 19h, le Paris Android User Group organise un meetup chez Radio France. Il paraît que le lieu est magnifique!
Je serai là en 2e partie pour vous parler de benchmarks dignes de confiance.
www.meetup.com/android-pari...
#AndroidDev #paris
It's available from Android 14 (SDK 34). You need to enable AIDL calls if you want to get it purely with Perfetto. Macrobenchmark enables it automatically.
September 22, 2025 at 8:43 AM
It's available from Android 14 (SDK 34). You need to enable AIDL calls if you want to get it purely with Perfetto. Macrobenchmark enables it automatically.
Yeah, I wasn't very happy about this advice, but back then there was no other choice.
Thankfully that's changed for some time. You can get information about the binder transaction from the binder reply.
It' automatically enabled in stable Macrobenchmark versions.
Thankfully that's changed for some time. You can get information about the binder transaction from the binder reply.
It' automatically enabled in stable Macrobenchmark versions.
September 19, 2025 at 1:05 PM
Yeah, I wasn't very happy about this advice, but back then there was no other choice.
Thankfully that's changed for some time. You can get information about the binder transaction from the binder reply.
It' automatically enabled in stable Macrobenchmark versions.
Thankfully that's changed for some time. You can get information about the binder transaction from the binder reply.
It' automatically enabled in stable Macrobenchmark versions.
On Android the modern standard for testing is Macrobenchmark and Microbenchmark frameworks. They can measure startup times, performance of all kinds of UI operations, and much more.
September 18, 2025 at 8:01 AM
On Android the modern standard for testing is Macrobenchmark and Microbenchmark frameworks. They can measure startup times, performance of all kinds of UI operations, and much more.
Benchmark 1.4.0 has improvements around:
- Microbenchmark: coroutines improvements!
- Macrobenchmark: Startup Insights!
- Baseline Profiles: support for multi-process apps
- TraceProcessor: split into its own module
developer.android.com/jetpack/andr...
- Microbenchmark: coroutines improvements!
- Macrobenchmark: Startup Insights!
- Baseline Profiles: support for multi-process apps
- TraceProcessor: split into its own module
developer.android.com/jetpack/andr...
Benchmark | Jetpack | Android Developers
developer.android.com
July 30, 2025 at 11:22 PM
Benchmark 1.4.0 has improvements around:
- Microbenchmark: coroutines improvements!
- Macrobenchmark: Startup Insights!
- Baseline Profiles: support for multi-process apps
- TraceProcessor: split into its own module
developer.android.com/jetpack/andr...
- Microbenchmark: coroutines improvements!
- Macrobenchmark: Startup Insights!
- Baseline Profiles: support for multi-process apps
- TraceProcessor: split into its own module
developer.android.com/jetpack/andr...
Solving flavor-related #benchmarking issues in #Android projects with multiple product flavors using Kotlin DSL. Learn how to handle tricky Gradle configurations for the macrobenchmark module. https://blog.stackademic.com/benchmarking-part-2-solving-androids-multiflavor-benchmarking-issues-in-kotli…
https://blog.stackademic.com/benchmarking-part-2-solving-androids-multiflavor-benchmarking-issues-in-kotlin-dsl-db98e6da06ba?source=rss----d1baaa8417a4---4
If you've tried adding Benchmarking to an Android project with multiple product flavors using Kotlin DSL, you've likely run into some…
blog.stackademic.com
June 30, 2025 at 10:31 PM
Solving flavor-related #benchmarking issues in #Android projects with multiple product flavors using Kotlin DSL. Learn how to handle tricky Gradle configurations for the macrobenchmark module. https://blog.stackademic.com/benchmarking-part-2-solving-androids-multiflavor-benchmarking-issues-in-kotli…
The latest update for #Bugfender includes "How to Use #GraphQL with Angular Using Apollo Client" and "#Android Macrobenchmark: Real-World Performance #Testing for Apps".
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
Bugfender
Bugfender is a modern remote logger tailor-made for mobile development.
opsmtrs.com
June 20, 2025 at 12:03 AM
The latest update for #Bugfender includes "How to Use #GraphQL with Angular Using Apollo Client" and "#Android Macrobenchmark: Real-World Performance #Testing for Apps".
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
The latest update for #Bugfender includes "#Android Macrobenchmark: Real-World #PerformanceTesting for Apps" and "Jetpack Compose State Management: A Guide for Android Developers".
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
Bugfender
Bugfender is a modern remote logger tailor-made for mobile development.
opsmtrs.com
June 13, 2025 at 4:49 PM
The latest update for #Bugfender includes "#Android Macrobenchmark: Real-World #PerformanceTesting for Apps" and "Jetpack Compose State Management: A Guide for Android Developers".
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
#App #crashreporting #logging https://opsmtrs.com/3F2zWgi
Mocking network responses in Macrobenchmark tests 👇
That is ok and I recommend it if you are benchmarking UI performance, layout, recomposition, scroll smoothness, frame times etc
In those scenarios loading times don’t provide relevant info, they’re actually noise and a source of flakiness.
That is ok and I recommend it if you are benchmarking UI performance, layout, recomposition, scroll smoothness, frame times etc
In those scenarios loading times don’t provide relevant info, they’re actually noise and a source of flakiness.
May 24, 2025 at 6:10 AM
Mocking network responses in Macrobenchmark tests 👇
That is ok and I recommend it if you are benchmarking UI performance, layout, recomposition, scroll smoothness, frame times etc
In those scenarios loading times don’t provide relevant info, they’re actually noise and a source of flakiness.
That is ok and I recommend it if you are benchmarking UI performance, layout, recomposition, scroll smoothness, frame times etc
In those scenarios loading times don’t provide relevant info, they’re actually noise and a source of flakiness.
I call macrobenchmark synthetic perf testing, when you control the environment to reduce confounding factors so code-induced perf changes are more apparent. That's in contrast to measuring perf in prod where the chaos makes regressions harder to find but is more representative of what users see.
May 10, 2025 at 6:47 PM
I call macrobenchmark synthetic perf testing, when you control the environment to reduce confounding factors so code-induced perf changes are more apparent. That's in contrast to measuring perf in prod where the chaos makes regressions harder to find but is more representative of what users see.
Since the goal is performance and avoiding performance regressions, I’d recommend macrobenchmark tests on CI to measure performance for the relevant flows.
May 7, 2025 at 8:08 PM
Since the goal is performance and avoiding performance regressions, I’d recommend macrobenchmark tests on CI to measure performance for the relevant flows.
Since then, we've had folks at Square write and run many Macrobenchmark tests, evaluating the impact of their changes with the script I provided here:
blog.p-y.wtf/a-script-to-...
blog.p-y.wtf/a-script-to-...
A script to compare two Macrobenchmarks runs
In Statistically Rigorous Android Macrobenchmarks, I laid out a methodology for rigorously comparing the outcome of two Jetpack Macrobenchmark runs. To summarize the article: Remove sources of variations until the distribution fits a normal distribu...
blog.p-y.wtf
February 25, 2025 at 5:22 AM
Since then, we've had folks at Square write and run many Macrobenchmark tests, evaluating the impact of their changes with the script I provided here:
blog.p-y.wtf/a-script-to-...
blog.p-y.wtf/a-script-to-...
If you're running Macrobenchmark tests to compute interaction latencies, you should check whether the distribution of results is a normal distribution.
If it's not normal, compare outlier traces with traces that are close to the median and you will find systemic perf issues!
Thread 🧵
#AndroidDev
If it's not normal, compare outlier traces with traces that are close to the median and you will find systemic perf issues!
Thread 🧵
#AndroidDev
February 25, 2025 at 5:22 AM
If you're running Macrobenchmark tests to compute interaction latencies, you should check whether the distribution of results is a normal distribution.
If it's not normal, compare outlier traces with traces that are close to the median and you will find systemic perf issues!
Thread 🧵
#AndroidDev
If it's not normal, compare outlier traces with traces that are close to the median and you will find systemic perf issues!
Thread 🧵
#AndroidDev
Facing some issue in Macrobenchmark.
I've installed the Main apk benchmark build variant and benchmark module apk manually on a device, then running the test using adb shell am instrument command.
The test is working fine for few iterations but after that it uninstall and reinstall the main apk.
I've installed the Main apk benchmark build variant and benchmark module apk manually on a device, then running the test using adb shell am instrument command.
The test is working fine for few iterations but after that it uninstall and reinstall the main apk.
December 26, 2024 at 6:42 PM
Facing some issue in Macrobenchmark.
I've installed the Main apk benchmark build variant and benchmark module apk manually on a device, then running the test using adb shell am instrument command.
The test is working fine for few iterations but after that it uninstall and reinstall the main apk.
I've installed the Main apk benchmark build variant and benchmark module apk manually on a device, then running the test using adb shell am instrument command.
The test is working fine for few iterations but after that it uninstall and reinstall the main apk.
2% improvement in Sidekiq's load test, pretty impressive considering this is a macrobenchmark:
json 2.7.2 500,000 jobs in 24.342253 sec, 20540 jobs/sec
json 2.9.1 500,000 jobs in 23.762558 sec, 21041 jobs/sec
And Sidekiq's json API usage: github.com/sidekiq/side...
json 2.7.2 500,000 jobs in 24.342253 sec, 20540 jobs/sec
json 2.9.1 500,000 jobs in 23.762558 sec, 21041 jobs/sec
And Sidekiq's json API usage: github.com/sidekiq/side...
sidekiq/lib/sidekiq.rb at 8648e796ff372acfce8afc2c32fa7d3ba77a449c · sidekiq/sidekiq
Simple, efficient background processing for Ruby. Contribute to sidekiq/sidekiq development by creating an account on GitHub.
github.com
December 18, 2024 at 5:31 PM
2% improvement in Sidekiq's load test, pretty impressive considering this is a macrobenchmark:
json 2.7.2 500,000 jobs in 24.342253 sec, 20540 jobs/sec
json 2.9.1 500,000 jobs in 23.762558 sec, 21041 jobs/sec
And Sidekiq's json API usage: github.com/sidekiq/side...
json 2.7.2 500,000 jobs in 24.342253 sec, 20540 jobs/sec
json 2.9.1 500,000 jobs in 23.762558 sec, 21041 jobs/sec
And Sidekiq's json API usage: github.com/sidekiq/side...
You can use this tool to visualise and compare the results of macrobenchmark
github.com/yogeshpaliya...
github.com/yogeshpaliya...
GitHub - yogeshpaliyal/benchmarkify: Benchmarkify is a web app that helps you visualize and analyze Android macrobenchmark results stored in JSON format.
Benchmarkify is a web app that helps you visualize and analyze Android macrobenchmark results stored in JSON format. - yogeshpaliyal/benchmarkify
github.com
December 6, 2024 at 8:09 AM
You can use this tool to visualise and compare the results of macrobenchmark
github.com/yogeshpaliya...
github.com/yogeshpaliya...
I'm a bit late with my #AndroidDev tip today!
Today's tip is to check out Baseline Profiles to speed up your app's initial launch: developer.android.com/topic/perfor...
And then remember to use the Macrobenchmark library to measure your startup times: developer.android.com/topic/perfor...
Today's tip is to check out Baseline Profiles to speed up your app's initial launch: developer.android.com/topic/perfor...
And then remember to use the Macrobenchmark library to measure your startup times: developer.android.com/topic/perfor...
November 27, 2024 at 10:33 PM
I'm a bit late with my #AndroidDev tip today!
Today's tip is to check out Baseline Profiles to speed up your app's initial launch: developer.android.com/topic/perfor...
And then remember to use the Macrobenchmark library to measure your startup times: developer.android.com/topic/perfor...
Today's tip is to check out Baseline Profiles to speed up your app's initial launch: developer.android.com/topic/perfor...
And then remember to use the Macrobenchmark library to measure your startup times: developer.android.com/topic/perfor...
For such a benchmark, which is very focused ("How long does it take to create a view model via Koin vs Hilt?"), I would probably use Microbenchmark rather than Macrobenchmark. The former will keep iterating until code is hot & the perf stable.
November 27, 2024 at 3:53 PM
For such a benchmark, which is very focused ("How long does it take to create a view model via Koin vs Hilt?"), I would probably use Microbenchmark rather than Macrobenchmark. The former will keep iterating until code is hot & the perf stable.
"Baseline Profiles Macrobenchmark" is for profiles, unrelated to this. Macrobenchmark proper actually does offer the ability to easily extract benchmark values for custom use, with TraceMetric: developer.android.com/reference/an...
November 27, 2024 at 3:53 PM
"Baseline Profiles Macrobenchmark" is for profiles, unrelated to this. Macrobenchmark proper actually does offer the ability to easily extract benchmark values for custom use, with TraceMetric: developer.android.com/reference/an...
The benchmark is run via a custom script, not microbenchmark nor macrobenchmark. These tools were written by experts and provide a lot of utility & checks that aren't in this script.
November 27, 2024 at 3:53 PM
The benchmark is run via a custom script, not microbenchmark nor macrobenchmark. These tools were written by experts and provide a lot of utility & checks that aren't in this script.
OK, I sent out jetc.dev Newsletter Issue #242! Math! Shared element transitions! Custom modifiers! Macrobenchmark! Adapting to platform-specific design systems! And... why does wrapContentSize() not wrap my content size?!? jetc.dev/issues/242 #JetpackCompose #AndroidDev
jetc.dev: Jetpack Compose Resources
Home of the jetc.dev Jetpack Compose newsletter! In the latest issue: Math! Shared element transitions! Custom modifiers! Macrobenchmark! Adapting to platform-specific design systems! And... why does ...
jetc.dev
November 26, 2024 at 1:53 PM
OK, I sent out jetc.dev Newsletter Issue #242! Math! Shared element transitions! Custom modifiers! Macrobenchmark! Adapting to platform-specific design systems! And... why does wrapContentSize() not wrap my content size?!? jetc.dev/issues/242 #JetpackCompose #AndroidDev
Added option to upload the Benchmark Results json file. (We do not store the file)
github.com/yogeshpaliya...
#AndroidDev #Macrobenchmark #OpenSource
github.com/yogeshpaliya...
#AndroidDev #Macrobenchmark #OpenSource
November 8, 2024 at 4:06 AM
Added option to upload the Benchmark Results json file. (We do not store the file)
github.com/yogeshpaliya...
#AndroidDev #Macrobenchmark #OpenSource
github.com/yogeshpaliya...
#AndroidDev #Macrobenchmark #OpenSource
🚀 Excited to introduce BenchMarkify ! 📊✨
Easily generate charts and tables from macrobenchmark results to visualize and optimize your app's performance. Check it out on GitHub and give it a star! 🌟
#AndroidDev #OpenSource #Benchmarking
github.yogeshpaliyal.com/benchmarkify/
Easily generate charts and tables from macrobenchmark results to visualize and optimize your app's performance. Check it out on GitHub and give it a star! 🌟
#AndroidDev #OpenSource #Benchmarking
github.yogeshpaliyal.com/benchmarkify/
November 4, 2024 at 4:41 PM
🚀 Excited to introduce BenchMarkify ! 📊✨
Easily generate charts and tables from macrobenchmark results to visualize and optimize your app's performance. Check it out on GitHub and give it a star! 🌟
#AndroidDev #OpenSource #Benchmarking
github.yogeshpaliyal.com/benchmarkify/
Easily generate charts and tables from macrobenchmark results to visualize and optimize your app's performance. Check it out on GitHub and give it a star! 🌟
#AndroidDev #OpenSource #Benchmarking
github.yogeshpaliyal.com/benchmarkify/
Had a ton of fun today writing a kotlin script that compares the result json files from 2 macrobenchmark runs.
@kaush.bsky.social your example repo was super useful, thanks!
@kaush.bsky.social your example repo was super useful, thanks!
May 17, 2023 at 11:52 PM
Had a ton of fun today writing a kotlin script that compares the result json files from 2 macrobenchmark runs.
@kaush.bsky.social your example repo was super useful, thanks!
@kaush.bsky.social your example repo was super useful, thanks!