Calculate · Explore · Learn

Computer Math Center

Understand binary, storage, internet speeds, programming, logic gates, screens, audio, hardware and networking. 100 tools with purposes, formulas and worked examples.

Simple mode uses starting defaults for extra scenario controls. Advanced exposes those controls where available. Change an entry, then press Calculate.

Find a tool

Choose a tool

Formula, calculation steps & example

All 100 tools: purposes and formulas

Decimal to binary — Convert integer values between base 10 and base 2.
Read place values using powers of 10; write the same integer using powers of 2.
Decimal to hexadecimal — Convert integer values between base 10 and base 16.
Read place values using powers of 10; write the same integer using powers of 16.
Decimal to octal — Convert integer values between base 10 and base 8.
Read place values using powers of 10; write the same integer using powers of 8.
Binary to decimal — Convert integer values between base 2 and base 10.
Read place values using powers of 2; write the same integer using powers of 10.
Hexadecimal to decimal — Convert integer values between base 16 and base 10.
Read place values using powers of 16; write the same integer using powers of 10.
Octal to decimal — Convert integer values between base 8 and base 10.
Read place values using powers of 8; write the same integer using powers of 10.
Binary to hexadecimal — Convert integer values between base 2 and base 16.
Read place values using powers of 2; write the same integer using powers of 16.
Hexadecimal to binary — Convert integer values between base 16 and base 2.
Read place values using powers of 16; write the same integer using powers of 2.
Any-base integer converter — Convert an integer between bases 2 through 36.
Value = sum(digit × input_base^place); express that value in output_base.
Two’s complement encode and decode — Encode a signed integer or decode a fixed-width bit pattern.
Encode negatives: 2^w + signed value; decode if high bit is 1: unsigned − 2^w.
Bits to bytes — Understand the size difference between the named units.
Output = input × 0.125.
Bytes to bits — Understand the size difference between the named units.
Output = input × 8.
GB to GiB — Understand the size difference between the named units.
Output = input × 0.9313225746154785.
GiB to GB — Understand the size difference between the named units.
Output = input × 1.073741824.
MB to MiB — Understand the size difference between the named units.
Output = input × 0.95367431640625.
MiB to MB — Understand the size difference between the named units.
Output = input × 1.048576.
How many files fit? — Count whole equal-sized files that fit in a storage budget.
Files = floor(GB × 1000 / MB).
Storage used and free — Measure available storage and utilization.
Free = total − used; used% = used/total × 100.
Compression ratio and savings — Compare original and compressed file sizes.
Ratio = original/compressed; saved% = (1 − compressed/original)×100.
RAID usable capacity model — Estimate raw array capacity for equal-size drives.
RAID 0: nS; full RAID 1 mirror: S; RAID 5: (n−1)S; RAID 6: (n−2)S; RAID 10: nS/2.
Mbps to MB per second — Learn why 100 Mbps corresponds to 12.5 MB/s before overhead.
MB/s = Mbps/8 × usable share.
MB per second to Mbps — Compare file-transfer rates with connection rates.
Mbps = MB/s × 8.
Download time — Estimate seconds and minutes to move a file.
Seconds = file_GB × 8000/(Mbps × usable share).
Upload time — Estimate seconds and minutes to move a file.
Seconds = file_GB × 8000/(Mbps × usable share).
Speed needed for a deadline — Solve the connection rate needed to transfer a file in time.
Required Mbps = GB×8000/(minutes×60×usable share).
Data cap runtime — Find how long constant-rate data use lasts.
Hours = GB×8000/(Mbps×3600).
Shared bandwidth per device — Explore a simplified equal-sharing connection.
Mbps/device = total usable Mbps/active devices.
Local backup time — Estimate a backup using a measured byte transfer rate.
Seconds = GB×1000/(MB/s).
Propagation latency lower bound — Model travel delay through a simplified link.
One-way milliseconds = distance/speed × 1000; round trip doubles it.
Throughput overhead effect — Explore how overhead changes useful transfer speed.
Useful Mbps = nominal × (1 − overhead/100).
Remainder and modulo — Compare signed remainder with a nonnegative modulo.
Remainder = a % b; nonnegative modulo = (remainder + b) % b.
Integer division with leftover — Split a nonnegative count into complete groups.
Complete groups = integer quotient; leftover = remainder.
Floor, ceiling and decimal rounding — Compare three rounding operations.
Floor ≤ value ≤ ceiling; rounded = round(value×10^d)/10^d.
Floating-point addition explorer — See a stored floating-point sum beside its rounded display.
Stored sum = IEEE-754 binary floating-point addition.
Unsigned integer range — See the exact range and count for a bit width.
Minimum = 0; maximum = 2^w−1; states=2^w.
Signed integer range — See a two’s complement signed range.
Minimum=−2^(w−1); maximum=2^(w−1)−1.
Bitwise AND — Combine two nonnegative integer bit patterns.
Output bit is 1 only if both input bits are 1.
Bitwise OR — Combine two nonnegative integer bit patterns.
Output bit is 1 if either input bit is 1.
Bitwise XOR — Combine two nonnegative integer bit patterns.
Output bit is 1 when the input bits differ.
Unsigned bit shift — Shift a nonnegative integer left or right without a width limit.
Left: n×2^k; right: floor(n/2^k).
AND truth table — Test inputs and inspect every possible input combination.
A AND B is 1 only when both are 1.
OR truth table — Test inputs and inspect every possible input combination.
A OR B is 1 when at least one is 1.
NOT truth table — Test inputs and inspect every possible input combination.
NOT A reverses 0 and 1.
XOR truth table — Test inputs and inspect every possible input combination.
A XOR B is 1 when they differ.
NAND truth table — Test inputs and inspect every possible input combination.
NAND = NOT(AND).
NOR truth table — Test inputs and inspect every possible input combination.
NOR = NOT(OR).
XNOR truth table — Test inputs and inspect every possible input combination.
XNOR = NOT(XOR).
Logical implication truth table — Test inputs and inspect every possible input combination.
A implies B = (NOT A) OR B.
Half-adder — Add two single bits and separate sum from carry.
Sum=A XOR B; carry=A AND B.
Full-adder — Add two bits and an incoming carry.
Sum=A XOR B XOR C; carry=floor((A+B+C)/2).
Image pixel count — Count all pixels in an image or screen.
Pixels = width×height; megapixels=pixels/1,000,000.
Aspect ratio reducer — Reduce screen dimensions to their simplest integer ratio.
Reduced ratio = width/GCD : height/GCD.
Proportional image resize — Find the height that preserves an image’s aspect ratio.
New height = original height×new width/original width.
Screen pixel density — Calculate pixels per inch from screen diagonal.
PPI = sqrt(width² + height²)/diagonal inches.
Uncompressed image size — Estimate pixel payload before compression.
Bytes = ceiling(width×height×bits per pixel/8).
RGB to hex color — Convert red, green and blue channel values to a color code.
Hex color = #RR GG BB; each channel is two hex digits.
Hex color to RGB — Read three RGB channel values from a hex code.
Parse RR, GG and BB as base-16 integers.
Color depth palette count — Count theoretical color codes for a bit depth.
Possible codes = 2^bits.
Image print dimensions — Convert image pixels to printed inches at a chosen density.
Width inches = width pixels/PPI; height inches = height pixels/PPI.
Screen buffer memory — Estimate multiple uncompressed screen buffers.
Memory bytes = width×height×bytes per pixel×buffer count.
Uncompressed PCM bitrate — Calculate raw audio sample bitrate.
Bits/second = sample rate×bits/sample×channels.
Uncompressed PCM recording size — Estimate an audio recording’s raw size.
Bytes = sample rate×bit depth×channels×minutes×60/8.
Video size from bitrate — Estimate compressed video size from its average total bitrate.
GB = Mbps×minutes×60/8000.
Recording time on a memory card — Estimate runtime from free capacity and recording bitrate.
Minutes = GB×8000/(Mbps×60).
Streaming data use — Estimate data use from an average stream bitrate.
GB = Mbps×hours×3600/8000.
Mono versus stereo PCM storage — Compare one and two uncompressed audio channels.
Mono bytes = rate×depth×seconds/8; stereo bytes = mono×2.
Audio bit-depth size comparison — Compare raw PCM storage at two bit depths.
Second payload = first payload×second depth/first depth.
Video frame count — Count frames at a constant frame rate.
Frames = FPS×minutes×60.
Uncompressed video bandwidth — Calculate raw pixel payload per second.
Bits/second = width×height×FPS×bits per pixel.
Video bitrate compression ratio — Compare an uncompressed bitrate with an encoded bitrate.
Ratio = raw bitrate/encoded bitrate.
Computer electricity budget — Estimate energy consumption and cost.
kWh = watts/1000×hours/day×days; cost=kWh×price.
UPS runtime energy estimate — Model battery runtime from usable energy and load.
Minutes = battery Wh×usable share/load watts×60.
FPS to frame time — Translate average frame rate to average milliseconds per frame.
Frame milliseconds = 1000/FPS.
CPU clock cycle duration — Calculate duration of one clock cycle.
Nanoseconds per cycle = 1/GHz.
Memory theoretical bandwidth — Model transfer rate times bus width and channel count.
GB/s = MT/s×bus bits/8×channels/1000.
RAM usage headroom — Compare installed RAM with estimated active usage.
Headroom = installed − active usage; usage%=active/installed×100.
Virtual machine RAM capacity — Count equal RAM allocations after reserving host memory.
VM count = floor((total − host reserve)/RAM per VM).
Storage upgrade headroom — Compare capacity with files and a reserve percentage.
Usable budget=capacity×(1−reserve%); headroom=usable budget−files.
Busy time utilization — Calculate the fraction of an interval spent busy.
Utilization%=busy time/interval×100.
Active pixel throughput — Calculate active pixels refreshed per second.
Active pixels/second = width×height×refresh rate.
IPv4 network address — Inspect a CIDR subnet using exact 32-bit address arithmetic.
Host bits=32−prefix; addresses=2^hostbits; network=address AND mask; upper=network+addresses−1.
IPv4 broadcast / upper address — Inspect a CIDR subnet using exact 32-bit address arithmetic.
Host bits=32−prefix; addresses=2^hostbits; network=address AND mask; upper=network+addresses−1.
IPv4 usable address range — Inspect a CIDR subnet using exact 32-bit address arithmetic.
Host bits=32−prefix; addresses=2^hostbits; network=address AND mask; upper=network+addresses−1.
CIDR to subnet mask — Inspect a CIDR subnet using exact 32-bit address arithmetic.
Host bits=32−prefix; addresses=2^hostbits; network=address AND mask; upper=network+addresses−1.
IPv4 address and host count — Inspect a CIDR subnet using exact 32-bit address arithmetic.
Host bits=32−prefix; addresses=2^hostbits; network=address AND mask; upper=network+addresses−1.
Subnet mask to CIDR — Convert a contiguous IPv4 subnet mask to prefix length.
Prefix length = number of leading 1 bits in a contiguous mask.
Subnet size for required hosts — Find a conventional subnet large enough for a host requirement.
Choose smallest h with 2^h−2 ≥ hosts; prefix=32−h.
Are two addresses in one subnet? — Compare two IPv4 addresses using one CIDR prefix.
Same subnet if (A AND mask) = (B AND mask).
IPv4 to integer — Convert a dotted IPv4 address to an unsigned integer.
Integer = a×256³ + b×256² + c×256 + d.
Integer to IPv4 — Decode an unsigned 32-bit integer into four octets.
Extract four base-256 digits from the 32-bit unsigned value.
UTF-8 text byte count — See why text characters can occupy different byte lengths.
UTF-8 size = sum of encoded bytes for each Unicode code point.
UTF-16 code units and bytes — Inspect JavaScript text length versus Unicode code points.
UTF-16 payload bytes = code units×2.
Unicode code point inspector — Inspect the numeric code point of one Unicode character.
Code point is the Unicode number assigned to a character.
ASCII text to binary bytes — Convert ASCII characters to eight-bit binary groups.
For each ASCII character, encode its integer code in eight binary digits.
Base64 output length — Calculate padded Base64 length from a byte count.
Padded Base64 characters = 4×ceiling(bytes/3).
Text to Base64 — Encode UTF-8 text in standard padded Base64.
Text → UTF-8 bytes → Base64 character groups.
Base64 to UTF-8 text — Decode a standard padded Base64 string.
Base64 symbols → bytes → UTF-8 text.
Character string combinations — Count fixed-length strings from an alphabet.
Combinations = alphabet_size^length.
Linear versus binary search bounds — Compare theoretical worst-case checks in a list.
Linear worst case=n; binary worst case=floor(log₂(n))+1.
Big-endian and little-endian bytes — Inspect two byte orders for the same unsigned integer.
Big-endian stores the most significant byte first; little-endian stores the least significant byte first.

Definitions and source references

Unit definitions and selected model references:

Save this math resource for later
Save to Pinterest