Results for “Precise numeric calculations”
Small, practical examples. Search by task or narrow things down below.
Interpolate between two calibration points
Arduino / C: Compute a fractional position before scaling to the output interval.
Arduino / C✓ Tested
Maintain a rolling sum in a circular window
Arduino / C: Subtract the overwritten sample before adding the next one.
Arduino / C✓ Tested
Parse a date without a time component
C#: DateOnly avoids attaching an irrelevant time or timezone to a calendar date.
C#✓ Tested
Parse a hexadecimal integer
PowerShell: Convert.ToInt32 accepts an explicit base for a validated hexadecimal string.
PowerShell✓ Tested
Parse an integer without throwing on invalid input
PowerShell: TryParse reports conversion success separately from its output value.
PowerShell✓ Tested
Parse an unsigned hexadecimal integer
C: Use an explicit base and check both range and full consumption.
C✓ Tested
Prevent output chatter with separate thresholds
Arduino / C: Use different on and off thresholds to retain state between them.
Arduino / C✓ Tested
Reject NaN and infinite values
C: isfinite detects whether a floating-point value is an ordinary finite number.
C✓ Tested
Reject one outlier with a median-of-three filter
Arduino / C: Sorting three values leaves the median in the middle.
Arduino / C✓ Tested
Represent a decimal quantity with a scaled integer
Arduino / C: Integer hundredths avoid floating-point storage for fixed-precision values.
Arduino / C✓ Tested
Return zero for a sum over no rows
SQL: SUM produces NULL for an empty input; COALESCE supplies the desired reporting default.
SQL✓ Tested
Return zero for a sum over no rows
SQL Server: SUM produces NULL for an empty input; COALESCE supplies the desired reporting default.
SQL Server✓ Tested