Results for “Precise numeric calculations”
Small, practical examples. Search by task or narrow things down below.
Compute a two-dimensional vector length
C: hypot avoids unnecessary intermediate overflow compared with squaring both inputs directly.
C✓ Tested
Compute an integer weighted average
SQL: Multiply before summing and guard a zero total weight with NULLIF.
SQL✓ Tested
Compute an integer weighted average
SQL Server: Multiply before summing and guard a zero total weight with NULLIF.
SQL Server✓ Tested
Compute an integer weighted average
Postgres: Multiply before summing and guard a zero total weight with NULLIF.
Postgres✓ Tested
Compute an integer weighted average
SQLite: Multiply before summing and guard a zero total weight with NULLIF.
SQLite✓ Tested
Compute beyond the fixed-width integer range
PowerShell: BigInteger keeps an exact result when ordinary integer types are too small.
PowerShell✓ Tested
Convert a supplied ADC count to volts
Arduino / C: Scale a ten-bit ADC count using an explicit reference voltage.
Arduino / C✓ Tested
Convert a timestamp to a different fixed offset
C#: ToOffset changes the displayed local time while retaining the same instant.
C#✓ Tested
Convert between integer bases
JavaScript: Supply a radix when parsing a nondecimal integer string.
JavaScript✓ Tested
Convert between integer bases
TypeScript: Supply a radix when parsing a nondecimal integer string.
TypeScript✓ Tested
Convert Celsius to Fahrenheit
Arduino / C: Use floating-point constants to avoid integer division.
Arduino / C✓ Tested
Convert Unix seconds into an aware UTC timestamp
PowerShell: DateTimeOffset.FromUnixTimeSeconds creates a value with an explicit UTC offset.
PowerShell✓ Tested