These tags (and some other similar ones) are known as "meta-tags": they describe certain aspects of the question that are independent of the mathematical subject the question is about, and they provide some information about what kinds of answer are sought. Their use is considered to be a necessary evil, and is generally discouraged. New users should not create new tags which do not describe aspects of mathematics.
The tags themselves
- reference-request should be used if you are looking for a specific reference. Questions of the form "What is the earliest known proof of X", or "Who is the first person to observe Y" fall under this category, as are questions of the form "Where can I find a proof of this Theorem/Inequality/Algebraic Identity?" But do note that quite often questions of this form can be answered with a Google or Wikipedia search, especially if you already known the "name" of the theorem or inequality you are looking for. What Math.SE excels in is the case when you know the statement of a theorem or inequality, but not its usual attribution. (In some sense, the reverse-lookup from the content to the name.)
- book-recommendation should be used if you need help choosing a book. Such questions should state the intended use of the book (independent study, course supplement, occasional reference, teaching a course) as well as any factors you consider important, such as the level of exposition, rigor, prerequisites, format or price.
- soft-question should be used if your question is not actually a mathematical question which admits a single correct answer, or if your question is one about mathematics or the practice of mathematics. Examples are "Why does my textbook solve problem X in this particular way, when I can also do it this other way?" or "Why does the definition of object Y requires property Z?"
The use of the (homework) tag is described in this FAQ item The (homework) tag has been removed.
- big-list should be used if your question asks for a list of books or examples.
- advice is used for "how to" questions, involving, for example, study techniques or career development career-development. Some of such questions may be more appropriate for Academia Stack Exchange.
The use of the tags
The most important thing about these meta tags is that they are meant to convey additional information about the question and not meant to stand alone as tags. In other words, you should never tag a question with only combinations of those four tags above (or other similar ones). For example, a reference-request tag should always be accompanied by a tag for the appropriate mathematical subject the question falls under.
Furthermore, Math.SE is intended, like other sites in the StackExchange network, to be a Q&A website. And we generally prefer questions that have actual answers, rather than questions that are prompts for discussions. (So no "meaning of life" questions here!) When you do ask a question that may not have a single correct answer (as is the case for all big-list questions and some soft-question questions), the moderators encourage you to flag the question for our attention, and we will convert the question to the Community Wiki format which is more suitable for those types of questions.