1. Why the Definite Integral Needs Extending
There are precisely two ways the Riemann framework can break:
The interval of integration extends to $+\infty$ or $-\infty$ (or both). For example:
$$\int_1^{\infty} \frac{1}{x^2}\,dx$$There is no finite upper endpoint, so the ordinary Riemann sum cannot be formed.
The integrand blows up at one or more points of the interval. For example:
$$\int_0^1 \frac{1}{\sqrt{x}}\,dx$$Here $f(x) = 1/\sqrt{x} \to \infty$ as $x \to 0^+$, so $f$ is unbounded on $[0,1]$.
The Unifying Idea: Limits of Proper Integrals
In both cases the strategy is the same: replace the problematic integral with a limit of ordinary (proper) integrals. The central question then becomes: does that limit exist and equal a finite number?
2. Type I — Infinite Limits of Integration
2.1 One Infinite Endpoint
Definition. If $f$ is continuous on $[a,\infty)$, then
$$\int_a^{\infty} f(x)\,dx = \lim_{b\to\infty} \int_a^b f(x)\,dx$$provided the limit exists. Similarly, if $f$ is continuous on $(-\infty, b]$:
$$\int_{-\infty}^{b} f(x)\,dx = \lim_{a\to -\infty} \int_a^b f(x)\,dx$$