Optimizing builds #3

Open
opened 2021-11-30 23:44:27 +00:00 by marceline-cramer · 2 comments
marceline-cramer commented 2021-11-30 23:44:27 +00:00 (Migrated from github.com)

Work that can be done to:

  • reduce the compile times for hopper
  • reduce the binary size for hopper

Dependency management

Disable unnecessary features in bigger crates.
(tokio is a big one)

indicatif and console import regex, which is slow... maybe use a lighter-weight terminal colors crates, and write our own progress bar? we'd need to make our own for the creeper face progress bar anyways

Replace larger, slower-compiling crates with more minimalistic ones.

Ideas:

More resources

Work that can be done to: - reduce the compile times for `hopper` - reduce the binary size for `hopper` # Dependency management Disable unnecessary features in bigger crates. (`tokio` is a big one) `indicatif` and `console` import `regex`, which is slow... maybe use a lighter-weight terminal colors crates, and write our own progress bar? we'd need to make our own for the creeper face progress bar anyways Replace larger, slower-compiling crates with more minimalistic ones. Ideas: - replace `tokio` with `smol` ([probably doesn't reduce compile times](https://www.reddit.com/r/rust/comments/hdb5m4/tips_for_faster_rust_compile_times/fvmayvh/), but might reduce binary size?) - replace `reqwest` with `attohttpc` or `ureq` # More resources - [endler.dev](https://endler.dev/2020/rust-compile-times/) - [blipjoy](https://blog.kodewerx.org/2020/06/the-rust-compiler-isnt-slow-we-are.html) - [the perf book](https://nnethercote.github.io/perf-book/compile-times.html)
marceline-cramer commented 2021-11-30 23:54:37 +00:00 (Migrated from github.com)

compile benches as of current main

time cargo build:

Executed in   68.64 secs    fish           external
   usr time  276.34 secs  657.00 micros  276.34 secs
   sys time   15.71 secs  128.00 micros   15.71 secs

ls -l target/release/hopper:

-rwxr-xr-x 2 mars mars 10060224 Nov 30 16:50 target/release/hopper

output of cargo bloat --time:

  Time Crate
19.08s tokio
17.97s syn
15.13s serde_derive
13.99s clap
13.83s regex_syntax
12.62s serde
12.09s http
11.58s encoding_rs
11.27s hopper
10.87s h2
10.82s openssl
10.10s hyper
10.08s cc
 9.60s regex
 9.10s reqwest
 8.74s proc_macro2
 8.21s futures_util
 6.30s serde_json
 6.30s libc
 6.04s unicode_normalization
 6.00s memchr
 5.39s mio
 5.33s structopt_derive
 5.32s proc_macro_error_attr
 5.05s toml
 4.95s tinyvec
 4.94s ipnet
 4.93s anyhow
 4.82s pkg_config
 4.76s unicode_bidi
 4.47s tracing_core
 4.40s signal_hook_registry
 4.20s build_script_main
 4.17s httparse
 4.13s rand_chacha
 4.04s futures_macro
 4.01s indexmap
 3.93s parking_lot_core
 3.85s console
 3.72s idna
 3.67s ryu
 3.58s bytes
 3.51s rand
 3.47s openssl_sys
 3.44s num_cpus
 3.42s autocfg
 3.39s socket2
 3.33s log
 3.31s parking_lot
 3.31s version_check
 3.24s unicode_segmentation
 3.01s url
 2.91s tokio_macros
 2.90s strsim
 2.88s futures_core
 2.88s dirs_sys
 2.86s quote
 2.82s futures_channel
 2.77s proc_macro_error
 2.77s native_tls
 2.74s futures_task
 2.65s indicatif
 2.64s base64
 2.51s hashbrown
 2.51s mime
 2.43s ppv_lite86
 2.42s dialoguer
 2.36s textwrap
 2.04s heck
 1.90s httpdate
 1.80s tracing
 1.74s getrandom
 1.74s tokio_util
 1.73s ansi_term
 1.70s openssl_probe
 1.54s percent_encoding
 1.49s rand_core
 1.46s directories
 1.41s form_urlencoded
 1.39s http_body
 1.33s lock_api
 1.24s smallvec
 1.20s once_cell
 1.18s want
 1.10s tempfile
 1.06s vec_map
 1.01s zeroize
 0.92s slab
 0.92s serde_urlencoded
 0.88s terminal_size
 0.80s itoa
 0.76s number_prefix
 0.76s unicode_xid
 0.71s atty
 0.66s confy
 0.63s unicode_width
 0.61s fnv
 0.61s instant
 0.58s hyper_tls
 0.57s tokio_native_tls
 0.56s futures_sink
 0.54s cfg_if
 0.53s pin_utils
 0.53s pin_project_lite
 0.52s foreign_types
 0.50s lazy_static
 0.49s scopeguard
 0.48s try_lock
 0.47s tower_service
 0.45s tinyvec_macros
 0.45s foreign_types_shared
 0.42s matches
 0.42s bitflags
 0.40s remove_dir_all
 0.35s cfg_if
 0.33s structopt
compile benches as of current `main` `time cargo build`: ``` Executed in 68.64 secs fish external usr time 276.34 secs 657.00 micros 276.34 secs sys time 15.71 secs 128.00 micros 15.71 secs ``` `ls -l target/release/hopper`: ``` -rwxr-xr-x 2 mars mars 10060224 Nov 30 16:50 target/release/hopper ``` output of `cargo bloat --time`: ``` Time Crate 19.08s tokio 17.97s syn 15.13s serde_derive 13.99s clap 13.83s regex_syntax 12.62s serde 12.09s http 11.58s encoding_rs 11.27s hopper 10.87s h2 10.82s openssl 10.10s hyper 10.08s cc 9.60s regex 9.10s reqwest 8.74s proc_macro2 8.21s futures_util 6.30s serde_json 6.30s libc 6.04s unicode_normalization 6.00s memchr 5.39s mio 5.33s structopt_derive 5.32s proc_macro_error_attr 5.05s toml 4.95s tinyvec 4.94s ipnet 4.93s anyhow 4.82s pkg_config 4.76s unicode_bidi 4.47s tracing_core 4.40s signal_hook_registry 4.20s build_script_main 4.17s httparse 4.13s rand_chacha 4.04s futures_macro 4.01s indexmap 3.93s parking_lot_core 3.85s console 3.72s idna 3.67s ryu 3.58s bytes 3.51s rand 3.47s openssl_sys 3.44s num_cpus 3.42s autocfg 3.39s socket2 3.33s log 3.31s parking_lot 3.31s version_check 3.24s unicode_segmentation 3.01s url 2.91s tokio_macros 2.90s strsim 2.88s futures_core 2.88s dirs_sys 2.86s quote 2.82s futures_channel 2.77s proc_macro_error 2.77s native_tls 2.74s futures_task 2.65s indicatif 2.64s base64 2.51s hashbrown 2.51s mime 2.43s ppv_lite86 2.42s dialoguer 2.36s textwrap 2.04s heck 1.90s httpdate 1.80s tracing 1.74s getrandom 1.74s tokio_util 1.73s ansi_term 1.70s openssl_probe 1.54s percent_encoding 1.49s rand_core 1.46s directories 1.41s form_urlencoded 1.39s http_body 1.33s lock_api 1.24s smallvec 1.20s once_cell 1.18s want 1.10s tempfile 1.06s vec_map 1.01s zeroize 0.92s slab 0.92s serde_urlencoded 0.88s terminal_size 0.80s itoa 0.76s number_prefix 0.76s unicode_xid 0.71s atty 0.66s confy 0.63s unicode_width 0.61s fnv 0.61s instant 0.58s hyper_tls 0.57s tokio_native_tls 0.56s futures_sink 0.54s cfg_if 0.53s pin_utils 0.53s pin_project_lite 0.52s foreign_types 0.50s lazy_static 0.49s scopeguard 0.48s try_lock 0.47s tower_service 0.45s tinyvec_macros 0.45s foreign_types_shared 0.42s matches 0.42s bitflags 0.40s remove_dir_all 0.35s cfg_if 0.33s structopt ```
emma added a new dependency 2022-12-18 01:41:07 +00:00
emma self-assigned this 2022-12-18 01:41:22 +00:00
╭ emma@noire-carnation
│ ~/s/T/m/hopper 
╰─[main]# unused-features analyze
A
B
[2022-12-24T04:06:34Z INFO  unused_features::subcommands::analyze] /home/emma/src/TiB/minecrust/hopper
[2022-12-24T04:06:36Z INFO  unused_features::subcommands::analyze] |===== Crate 'hopper' =====|
[2022-12-24T04:06:36Z INFO  unused_features::subcommands::analyze] Start pruning feature flags. The process will recompile the project 39 times.
[2022-12-24T04:06:36Z INFO  unused_features::subcommands::analyze] [0.0%]: ==== Dependency 'anyhow', removing 1 flags =====
[2022-12-24T04:06:36Z INFO  unused_features::subcommands::analyze] [0.0%]: Prune 'std' feature flag from 'anyhow'
[2022-12-24T04:06:57Z INFO  unused_features::subcommands::analyze] [10.0%]: ==== Dependency 'serde_json', removing 1 flags =====
[2022-12-24T04:06:57Z INFO  unused_features::subcommands::analyze] [10.0%]: Prune 'std' feature flag from 'serde_json'
[2022-12-24T04:06:59Z INFO  unused_features::subcommands::analyze] [20.0%]: ==== Dependency 'dialoguer', removing 2 flags =====
[2022-12-24T04:06:59Z INFO  unused_features::subcommands::analyze] [20.0%]: Prune 'password' feature flag from 'dialoguer'
[2022-12-24T04:07:01Z INFO  unused_features::subcommands::analyze] [25.0%]: Prune 'editor' feature flag from 'dialoguer'
[2022-12-24T04:07:03Z INFO  unused_features::subcommands::analyze] [30.0%]: ==== Dependency 'env_logger', removing 4 flags =====
[2022-12-24T04:07:03Z INFO  unused_features::subcommands::analyze] [30.0%]: Prune 'humantime' feature flag from 'env_logger'
[2022-12-24T04:07:05Z INFO  unused_features::subcommands::analyze] [32.5%]: Prune 'atty' feature flag from 'env_logger'
[2022-12-24T04:07:06Z INFO  unused_features::subcommands::analyze] [35.0%]: Prune 'regex' feature flag from 'env_logger'
[2022-12-24T04:07:10Z INFO  unused_features::subcommands::analyze] [37.5%]: Prune 'termcolor' feature flag from 'env_logger'
[2022-12-24T04:07:12Z INFO  unused_features::subcommands::analyze] [40.0%]: ==== Dependency 'tokio', removing 13 flags =====
[2022-12-24T04:07:12Z INFO  unused_features::subcommands::analyze] [40.0%]: Prune 'time' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [40.8%]: Prune 'signal' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [41.5%]: Prune 'rt-multi-thread' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [42.3%]: Prune 'process' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [43.1%]: Prune 'parking_lot' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [43.8%]: Prune 'sync' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [44.6%]: Prune 'fs' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [45.4%]: Prune 'macros' feature flag from 'tokio'
[2022-12-24T04:07:17Z INFO  unused_features::subcommands::analyze] [46.2%]: Prune 'full' feature flag from 'tokio'
[2022-12-24T04:07:27Z INFO  unused_features::subcommands::analyze] [46.9%]: Prune 'rt' feature flag from 'tokio'
[2022-12-24T04:07:39Z INFO  unused_features::subcommands::analyze] [47.7%]: Prune 'net' feature flag from 'tokio'
[2022-12-24T04:07:39Z INFO  unused_features::subcommands::analyze] [48.5%]: Prune 'io-util' feature flag from 'tokio'
[2022-12-24T04:07:39Z INFO  unused_features::subcommands::analyze] [49.2%]: Prune 'io-std' feature flag from 'tokio'
[2022-12-24T04:07:39Z INFO  unused_features::subcommands::analyze] [50.0%]: ==== Dependency 'clap', removing 6 flags =====
[2022-12-24T04:07:39Z INFO  unused_features::subcommands::analyze] [50.0%]: Prune 'suggestions' feature flag from 'clap'
[2022-12-24T04:07:43Z INFO  unused_features::subcommands::analyze] [51.7%]: Prune 'color' feature flag from 'clap'
[2022-12-24T04:07:55Z INFO  unused_features::subcommands::analyze] [53.3%]: Prune 'derive' feature flag from 'clap'
[2022-12-24T04:07:59Z INFO  unused_features::subcommands::analyze] [55.0%]: Prune 'clap_derive' feature flag from 'clap'
[2022-12-24T04:07:59Z INFO  unused_features::subcommands::analyze] [56.7%]: Prune 'once_cell' feature flag from 'clap'
[2022-12-24T04:07:59Z INFO  unused_features::subcommands::analyze] [58.3%]: Prune 'std' feature flag from 'clap'
[2022-12-24T04:07:59Z INFO  unused_features::subcommands::analyze] [60.0%]: ==== Dependency 'reqwest', removing 4 flags =====
[2022-12-24T04:07:59Z INFO  unused_features::subcommands::analyze] [60.0%]: Prune 'json' feature flag from 'reqwest'
[2022-12-24T04:08:02Z INFO  unused_features::subcommands::analyze] [62.5%]: Prune 'stream' feature flag from 'reqwest'
[2022-12-24T04:08:04Z INFO  unused_features::subcommands::analyze] [65.0%]: Prune 'serde_json' feature flag from 'reqwest'
[2022-12-24T04:08:08Z INFO  unused_features::subcommands::analyze] [67.5%]: Prune 'default-tls' feature flag from 'reqwest'
[2022-12-24T04:08:11Z INFO  unused_features::subcommands::analyze] [70.0%]: ==== Dependency 'serde', removing 3 flags =====
[2022-12-24T04:08:11Z INFO  unused_features::subcommands::analyze] [70.0%]: Prune 'std' feature flag from 'serde'
[2022-12-24T04:08:22Z INFO  unused_features::subcommands::analyze] [73.3%]: Prune 'serde_derive' feature flag from 'serde'
[2022-12-24T04:08:23Z INFO  unused_features::subcommands::analyze] [76.7%]: Prune 'derive' feature flag from 'serde'
[2022-12-24T04:08:28Z INFO  unused_features::subcommands::analyze] [80.0%]: ==== Dependency 'futures-util', removing 3 flags =====
[2022-12-24T04:08:28Z INFO  unused_features::subcommands::analyze] [80.0%]: Prune 'async-await' feature flag from 'futures-util'
[2022-12-24T04:08:40Z INFO  unused_features::subcommands::analyze] [83.3%]: Prune 'std' feature flag from 'futures-util'
[2022-12-24T04:08:50Z INFO  unused_features::subcommands::analyze] [86.7%]: Prune 'async-await-macro' feature flag from 'futures-util'
[2022-12-24T04:09:07Z INFO  unused_features::subcommands::analyze] [90.0%]: ==== Dependency 'console', removing 2 flags =====
[2022-12-24T04:09:07Z INFO  unused_features::subcommands::analyze] [90.0%]: Prune 'unicode-width' feature flag from 'console'
[2022-12-24T04:09:09Z INFO  unused_features::subcommands::analyze] [95.0%]: Prune 'ansi-parsing' feature flag from 'console'
[2022-12-24T04:09:09Z INFO  unused_features::report] Written json report to /home/emma/src/TiB/minecrust/hopper/report.json.
[2022-12-24T04:09:09Z INFO  unused_features] Finished the process
``` ╭ emma@noire-carnation │ ~/s/T/m/hopper ╰─[main]# unused-features analyze A B [2022-12-24T04:06:34Z INFO unused_features::subcommands::analyze] /home/emma/src/TiB/minecrust/hopper [2022-12-24T04:06:36Z INFO unused_features::subcommands::analyze] |===== Crate 'hopper' =====| [2022-12-24T04:06:36Z INFO unused_features::subcommands::analyze] Start pruning feature flags. The process will recompile the project 39 times. [2022-12-24T04:06:36Z INFO unused_features::subcommands::analyze] [0.0%]: ==== Dependency 'anyhow', removing 1 flags ===== [2022-12-24T04:06:36Z INFO unused_features::subcommands::analyze] [0.0%]: Prune 'std' feature flag from 'anyhow' [2022-12-24T04:06:57Z INFO unused_features::subcommands::analyze] [10.0%]: ==== Dependency 'serde_json', removing 1 flags ===== [2022-12-24T04:06:57Z INFO unused_features::subcommands::analyze] [10.0%]: Prune 'std' feature flag from 'serde_json' [2022-12-24T04:06:59Z INFO unused_features::subcommands::analyze] [20.0%]: ==== Dependency 'dialoguer', removing 2 flags ===== [2022-12-24T04:06:59Z INFO unused_features::subcommands::analyze] [20.0%]: Prune 'password' feature flag from 'dialoguer' [2022-12-24T04:07:01Z INFO unused_features::subcommands::analyze] [25.0%]: Prune 'editor' feature flag from 'dialoguer' [2022-12-24T04:07:03Z INFO unused_features::subcommands::analyze] [30.0%]: ==== Dependency 'env_logger', removing 4 flags ===== [2022-12-24T04:07:03Z INFO unused_features::subcommands::analyze] [30.0%]: Prune 'humantime' feature flag from 'env_logger' [2022-12-24T04:07:05Z INFO unused_features::subcommands::analyze] [32.5%]: Prune 'atty' feature flag from 'env_logger' [2022-12-24T04:07:06Z INFO unused_features::subcommands::analyze] [35.0%]: Prune 'regex' feature flag from 'env_logger' [2022-12-24T04:07:10Z INFO unused_features::subcommands::analyze] [37.5%]: Prune 'termcolor' feature flag from 'env_logger' [2022-12-24T04:07:12Z INFO unused_features::subcommands::analyze] [40.0%]: ==== Dependency 'tokio', removing 13 flags ===== [2022-12-24T04:07:12Z INFO unused_features::subcommands::analyze] [40.0%]: Prune 'time' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [40.8%]: Prune 'signal' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [41.5%]: Prune 'rt-multi-thread' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [42.3%]: Prune 'process' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [43.1%]: Prune 'parking_lot' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [43.8%]: Prune 'sync' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [44.6%]: Prune 'fs' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [45.4%]: Prune 'macros' feature flag from 'tokio' [2022-12-24T04:07:17Z INFO unused_features::subcommands::analyze] [46.2%]: Prune 'full' feature flag from 'tokio' [2022-12-24T04:07:27Z INFO unused_features::subcommands::analyze] [46.9%]: Prune 'rt' feature flag from 'tokio' [2022-12-24T04:07:39Z INFO unused_features::subcommands::analyze] [47.7%]: Prune 'net' feature flag from 'tokio' [2022-12-24T04:07:39Z INFO unused_features::subcommands::analyze] [48.5%]: Prune 'io-util' feature flag from 'tokio' [2022-12-24T04:07:39Z INFO unused_features::subcommands::analyze] [49.2%]: Prune 'io-std' feature flag from 'tokio' [2022-12-24T04:07:39Z INFO unused_features::subcommands::analyze] [50.0%]: ==== Dependency 'clap', removing 6 flags ===== [2022-12-24T04:07:39Z INFO unused_features::subcommands::analyze] [50.0%]: Prune 'suggestions' feature flag from 'clap' [2022-12-24T04:07:43Z INFO unused_features::subcommands::analyze] [51.7%]: Prune 'color' feature flag from 'clap' [2022-12-24T04:07:55Z INFO unused_features::subcommands::analyze] [53.3%]: Prune 'derive' feature flag from 'clap' [2022-12-24T04:07:59Z INFO unused_features::subcommands::analyze] [55.0%]: Prune 'clap_derive' feature flag from 'clap' [2022-12-24T04:07:59Z INFO unused_features::subcommands::analyze] [56.7%]: Prune 'once_cell' feature flag from 'clap' [2022-12-24T04:07:59Z INFO unused_features::subcommands::analyze] [58.3%]: Prune 'std' feature flag from 'clap' [2022-12-24T04:07:59Z INFO unused_features::subcommands::analyze] [60.0%]: ==== Dependency 'reqwest', removing 4 flags ===== [2022-12-24T04:07:59Z INFO unused_features::subcommands::analyze] [60.0%]: Prune 'json' feature flag from 'reqwest' [2022-12-24T04:08:02Z INFO unused_features::subcommands::analyze] [62.5%]: Prune 'stream' feature flag from 'reqwest' [2022-12-24T04:08:04Z INFO unused_features::subcommands::analyze] [65.0%]: Prune 'serde_json' feature flag from 'reqwest' [2022-12-24T04:08:08Z INFO unused_features::subcommands::analyze] [67.5%]: Prune 'default-tls' feature flag from 'reqwest' [2022-12-24T04:08:11Z INFO unused_features::subcommands::analyze] [70.0%]: ==== Dependency 'serde', removing 3 flags ===== [2022-12-24T04:08:11Z INFO unused_features::subcommands::analyze] [70.0%]: Prune 'std' feature flag from 'serde' [2022-12-24T04:08:22Z INFO unused_features::subcommands::analyze] [73.3%]: Prune 'serde_derive' feature flag from 'serde' [2022-12-24T04:08:23Z INFO unused_features::subcommands::analyze] [76.7%]: Prune 'derive' feature flag from 'serde' [2022-12-24T04:08:28Z INFO unused_features::subcommands::analyze] [80.0%]: ==== Dependency 'futures-util', removing 3 flags ===== [2022-12-24T04:08:28Z INFO unused_features::subcommands::analyze] [80.0%]: Prune 'async-await' feature flag from 'futures-util' [2022-12-24T04:08:40Z INFO unused_features::subcommands::analyze] [83.3%]: Prune 'std' feature flag from 'futures-util' [2022-12-24T04:08:50Z INFO unused_features::subcommands::analyze] [86.7%]: Prune 'async-await-macro' feature flag from 'futures-util' [2022-12-24T04:09:07Z INFO unused_features::subcommands::analyze] [90.0%]: ==== Dependency 'console', removing 2 flags ===== [2022-12-24T04:09:07Z INFO unused_features::subcommands::analyze] [90.0%]: Prune 'unicode-width' feature flag from 'console' [2022-12-24T04:09:09Z INFO unused_features::subcommands::analyze] [95.0%]: Prune 'ansi-parsing' feature flag from 'console' [2022-12-24T04:09:09Z INFO unused_features::report] Written json report to /home/emma/src/TiB/minecrust/hopper/report.json. [2022-12-24T04:09:09Z INFO unused_features] Finished the process ```
emma added reference rewrite 2023-03-23 02:57:02 +00:00
emma changed reference from rewrite to main 2023-03-23 02:57:09 +00:00
Sign in to join this conversation.
No description provided.