【Matter】 ,  【substance】 ,  【material】 ,  【stuff】  are comparable when they mean what goes into the makeup or forms the being of a thing whether physical or not.

In the relevant sense  【matter】 basically denotes that of which all physical objects are made, but to the physical scientist this may imply the component of the observable universe that includes among its properties extension, inertia, and gravitation and is held to consist ultimately of relatively few kinds of elementary particles, to be convertible into energy, and together with energy to form the basis of objective phenomena while to the philosopher it tends to imply an unordered 【material】 substratum distinguishable on the one hand from immaterial qualities (as spirit or energy) and on the other from form or formed bodies.

In more general use  【matter】 often specifically applies to a particular portion or kind of physical 【matter】 .

【Substance】 usually implies a particular kind of 【matter】 and often one of known chemical or physical nature or it may distinctively suggest the particular 【matter】 or kind of 【matter】 that enters into the composition of something or gives it its characteristic properties.

【Material】 applies basically to 【matter】 or 【substance】 as a constituent of physical and especially of made things.

But often it subordinates physical nature to the fact of being made and then may imply the idea of actuality or a basis in actuality rather than physical 【substance】 ; thus, raw materials are usually physical substances capable of refinement or manufacture but they also may be events, ideas, or facts capable of further use (as in literary creation).

【Stuff】  may replace  【material】  or  【substance】 in reference to constituent materials or substances, but more often it refers to all the parts, parcels, objects, or items that make up an aggregate or a whole; in both cases it is likely to imply indeterminateness and suggest vaguely if at all the nature of the constituent materials or parts.

Sometimes, distinctively,  【stuff】 carries an inherent implication of inferiority.