Development
Here are some development related topiscs.
How to run JMH benchmarks
Follow this steps:
* Compile whole project and create pacakge containing all benchmarks data
* Go to jmh-benchmarks
* Execute it, with temp directory in target directory
Specific JMH benchmark class could be run:
result could look like:
Benchmark Mode Cnt Score Error Units
SequentialFileReadingBenchmark.test_with_buffer_01KB avgt 4 70.747 ± 42.480 ms/op
SequentialFileReadingBenchmark.test_with_buffer_02KB avgt 4 60.009 ± 52.899 ms/op
SequentialFileReadingBenchmark.test_with_buffer_04KB avgt 4 51.254 ± 30.112 ms/op
SequentialFileReadingBenchmark.test_with_buffer_08KB avgt 4 48.600 ± 28.892 ms/op
SequentialFileReadingBenchmark.test_with_buffer_16KB avgt 4 48.471 ± 25.665 ms/op
SequentialFileReadingBenchmark.test_with_buffer_32KB avgt 4 45.256 ± 24.986 ms/op
SequentialFileReadingBenchmark.test_with_buffer_64KB avgt 4 45.204 ± 24.867 ms/op
SequentialFileWritingBenchmark.test_with_buffer_01KB avgt 4 238.075 ± 75.311 ms/op
SequentialFileWritingBenchmark.test_with_buffer_02KB avgt 4 271.272 ± 64.747 ms/op
SequentialFileWritingBenchmark.test_with_buffer_04KB avgt 4 276.001 ± 45.815 ms/op
SequentialFileWritingBenchmark.test_with_buffer_08KB avgt 4 352.189 ± 1140.814 ms/op
SequentialFileWritingBenchmark.test_with_buffer_16KB avgt 4 258.806 ± 44.693 ms/op
SequentialFileWritingBenchmark.test_with_buffer_32KB avgt 4 276.246 ± 135.019 ms/op
SequentialFileWritingBenchmark.test_with_buffer_64KB avgt 4 275.944 ± 128.835 ms/op
When some JMH benchmark class is changed command mvn package have to be run.
Possible problems
Generally JMH is quite fragile. Small changes broke JMH benchmark execution. Usually helps rebuild project and start again as described above.
Load test
Runnign JVM should be inspected with some profiller. For profilling is usefull to hae long running task to watch it. Go to project load-test. Following command show all optional parameters:
Theer are two main supported operations. First is data generating. It's could be usefull to place in java profilling agent. It could look like:
java \
-agentpath:/Applications/YourKit-Java-Profiler-2023.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=exceptions=disable,delay=10000,listen=all \
-jar target/load-test.jar com.coroptis.index.loadtest.Main \
--write \
--directory /Volumes/LaCie/test/ \
--count 5_000_000_000 \
--max-number-of-keys-in-cache 5_000_000 \
--max-number-of-keys-in-segment 10_000_000 \
--max-number-of-keys-in-segment-cache 500_000 \
--max-number-of-keys-in-segment-cache-during-flushing 2_000_000 \
--max-number-of-keys-in-segment-index-page 1_000 \
--bloom-filter-index-size-in-bytes 10_000_000 \
--bloom-filter-number-of-hash-functions 2
It will generate 210 GB of testing data. Furst search test can be performed like this:
java \
-agentpath:/Applications/YourKit-Java-Profiler-2023.9.app/Contents/Resources/bin/mac/libyjpagent.dylib=exceptions=disable,delay=10000,listen=all \
-jar target/load-test.jar com.coroptis.index.loadtest.Main \
--search \
--directory /Volumes/LaCie/test/ \
--count 5_000_000_000 \
--max-key 5_000_000_000 \
--max-number-of-keys-in-cache 5_000_000 \
--max-number-of-keys-in-segment 10_000_000 \
--max-number-of-keys-in-segment-cache 500_000 \
--max-number-of-keys-in-segment-cache-during-flushing 2_000_000 \
--max-number-of-keys-in-segment-index-page 1_000 \
--bloom-filter-index-size-in-bytes 10_000_000 \
--bloom-filter-number-of-hash-functions 2
Development
Mockito requires reflective access to non-public parts in a Java module. It could be manually open by passing following parameter as jvm parameter:
How to get segment disk size
On apple try: