Until now, soft X-ray (sX-ray) radiation with a low pulse duration in the picosecond range for the study of magnetism was exclusive to large-scale facilities, such as synchrotron radiation facilities or free electron lasers. However, this thesis demonstrates the capabilities of using a laser-driven plasma X-ray source (LPXS) for the study of magnetic order across the soft X-ray transition metal and rare earth edges at few-ps time resolution. Thus far, outside of a large-scale facility, higher harmonic generation was able to provide pulses with femto- or even attosecond duration, but efficiently only with photon energies reaching the upper end of the water window at ≈500 eV. However, for resolving element- selective magnetic order in complex heterostructures, sX-ray radiation in the range from 500 eV to 1500 eV is vital. In addition, the shorter wavelengths allow for a higher spatial resolution down to single nanometres. A LPXS on the other hand, generates intense plasmas that emit broadband soft X-ray pulses across the entire soft X-ray range in a single pulse. Their accompanying low spatial coherence and random polarisation make LPXSs seemingly unsuitable for the study of magnetic order. To disprove this assumption, this thesis presents four experimental techniques, based on different magneto-optical effects, which enable the study of magnetic order with a high signal- to-noise ratio in no time. Namely, these are the scattering techniques resonant magnetic X-ray scattering and small-angle X-ray scattering, as well as white-light spectroscopy techniques, based on the well-known X-ray magnetic circular dichroism and the newly-uncovered X-ray magnetic circular birefringence-effect. For this, a setup was constructed, which utilises a stack of reflective zone plate optics that collect, disperse, and focus different parts of the spectrum from a LPXS to a sample position in an experimental chamber. An ultrashort pump laser pulse allows for the excitation of the sample at a time resolution of <10 ps. The laboratory environment enables fine control over the sample environment and provides the ability to quickly adapt even more detection schemes. The high stability of the setup allows for the study of the influence of these and other external parameters on the sample system in great detail with high statistics. The findings about the setup in general, and the four techniques in particular, were already or are about to be submitted for publication in five first-author publications in peer-reviewed journals. These experiments, previously impossible in a laboratory environment, are now readily available for the study of magnetic order and the underlying coupling and transport phenomena on ultrafast timescales. Furthermore, these findings demonstrate the potential for using these LPXSs in other fields beyond ultrafast magnetism.