Results for “Precise numeric calculations”
Small, practical examples. Search by task or narrow things down below.
Reject NaN and infinite values
PowerShell: Double.IsFinite excludes numeric values unsuitable for ordinary bounded calculations.
PowerShell✓ Tested
Apply a linear sensor calibration
Arduino / C: Convert a raw reading using an offset and scale measured for the sensor.
Arduino / C✓ Tested
Calculate a page count with ceiling division
JavaScript: Use Math.ceil for a nonnegative item count and a positive page size.
JavaScript✓ Tested
Calculate a page count with ceiling division
TypeScript: Use Math.ceil for a nonnegative item count and a positive page size.
TypeScript✓ Tested
Calculate a page count without an overflowing addition
C: Divide first and add one only when a nonzero remainder remains.
C✓ Tested
Calculate an integer beyond the 64-bit range
C#: Use BigInteger for arbitrary-precision integer arithmetic.
C#✓ Tested
Calculate each group’s integer percentage of the total
SQL: Multiply before division and protect against a zero denominator.
SQL✓ Tested
Calculate each group’s integer percentage of the total
SQL Server: Multiply before division and protect against a zero denominator.
SQL Server✓ Tested
Calculate each group’s integer percentage of the total
Postgres: Multiply before division and protect against a zero denominator.
Postgres✓ Tested
Calculate each group’s integer percentage of the total
SQLite: Multiply before division and protect against a zero denominator.
SQLite✓ Tested
Calculate the number of calendar days between dates
C#: Subtract DayNumber values when the inputs are calendar dates.
C#✓ Tested
Calculate the number of required pages
Python: Use integer arithmetic to round a nonnegative count up to a positive page size.
Python✓ Tested