We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Java, How Fast Can You Parse 1 Billion Rows of Weather Data? • Roy van Rijn • GOTO 2024
Java developer Roy van Rijn presents his GOTO 2024 talk on achieving fast parsing of vast weather data sets, exploring techniques such as memory-mapped files, branchless programming, and native compilation.
- Java’s garbage collection can slow down processing, especially when dealing with large amounts of data.
- Using a memory-mapped file can significantly speed up processing.
- Branchless programming can be used to optimize code and reduce branches, making it faster.
-
Using
Unsafeclass can allow for low-level memory access and manipulation, making it possible to create a hash table without using objects. - Native compilation and binary compilation can further optimize code.
- Multiplying every bit by 10 can be used to quickly parse temperature values.
- Using a single long value to store the temperature can reduce memory allocation and improve performance.
- SWAR (Single Write Array Read) method can be used to optimize code and reduce overhead.
- Using JSON-LD for cross-platform communication can be more efficient than SOAP.
- Observing the behavior of the JVM can be helpful in understanding how it works.
- Using a console app and having it print out the results can be a way to view and test the output.
- Debugging and testing is important to ensure the program is working correctly.
- Hash collisions can occur and result in slower performance.
- The JVM (Java Virtual Machine) is a layer between the Java code and the underlying computer hardware.
- Compiler optimization and predictive vectorization can also improve performance.
- Using a for loop instead of recursion can sometimes be more efficient.
- Using parallel processing can take advantage of multi-core processors.
- Java’s Byte Buffer can be used to store and retrieve bytes efficiently.
- The JVM can optimize code, but this is not always the case.
- The GraalVM is a just-in-time compiler that can run native code.
- The Oracle GraalVM is a fixed-size array implementation.
- The register allocation refers to how the JVM assigns memory for variables.
- The JVM is a layer between the Java code and the underlying computer hardware.
- Compiler optimization can improve performance.
- The garbage collector should clear up the memory after use.
- Un stunned and in agreement with you, you’re my colleague, they’re my colleagues.