The central theorem in the finite group case is the Frobenius reciprocity theorem. It is stated in terms of another construction of representations, the restriction map (which is a functor): any linear representation of G, as K[G]-module where K[G] is the group ring of G over a field K, is also a K[H]-module. The theorem states that, given representations of G and σ of H, the space of G-intertwining maps from ρ to Ind(σ) has the same dimension as that of the H-intertwining maps from Res(ρ) to σ. (Here Res stands for restricted representation, and Ind for induced representation.) It is useful (in the typical case of non-modular representations, anyway - say with K = C) for computing the decomposition of the induced representation: we can do calculations on the side of H, which is the 'small' group.
In fact, anachronistically, we can recognise that this theorem shows that Res and Ind are adjoint functors. The content of that statement is more than the dimensions: it requires that the isomorphism of vector spaces of intertwining maps be natural, in the sense of category theory. It actually suggests that induced representation can in this case be defined by means of the adjunction. That's not the only way to do it - and perhaps not the only helpful way - but it means that the theory will not be ad hoc in its start.
One can therefore makes the reciprocity theorem the way to defining the induced representation. There is another way, suggested by the permutation examples of the introductory paragraph. The induced representation Ind(σ) should be realized as a space of functions on G transforming under H according to the representation σ. Therefore if σ acts on the vector space V, we should look at V-valued functions on G on which H acts via σ (this must be said carefully with explicit talk about left- and right-actions). This approach allows the induced representation to be a kind of free module construction.
The two approaches outlined above can be reconciled in the case of finite groups, by using the tensor product with K[G] as a K[H]-module. There is a third and classical approach, of simply writing down the character (trace) of the induced representation, in terms of conjugation in G of elements g into H.
In more general terms, the reciprocity theorem isn't available in generality for representations of topological groups; and the character formulas are also subject to some analytical problems. The second definition, on the other hand, is a major theme in harmonic analysis, in generality. It is adapted to the theory of vector bundles, for example.