Rare-earth ion doped crystals are promising systems for quantum communication and quantum information processing. In particular, paramagnetic rare-earth centres can be utilized to realize quantum coherent interfaces simultaneously for optical and microwave photons. In this article, we study hyperfine and magnetic properties of a Y$_2$SiO$_5$ crystal doped with $^{171}$Yb$^{3+}$ ions. This isotope is particularly interesting since it is the only rare--earth ion having electronic spin $S=\frac{1}{2}$ and nuclear spin $I=\frac{1}{2}$, which results in the simplest possible hyperfine level structure. In this work we determine the hyperfine tensors for the ground and excited states on the optical $^2$F$_{7/2}(0) \longleftrightarrow ^2$F$_{5/2}$(0) transition by combining spectral holeburning and optically detected magnetic resonance techniques. The resulting spin Hamiltonians correctly predict the magnetic-field dependence of all observed optical-hyperfine transitions, from zero applied field to the high-field regime where the Zeeman interaction is dominating. Using the optical absorption spectrum we can also determine the order of the hyperfine levels in both states. These results pave the way for realizing solid-state optical and microwave quantum memories based on a $^{171}$Yb$^{3+}$:Y$_2$SiO$_5$ crystal.