It depends. If you have credit card debt, paying that off is the best. Other debt would need to be compared to your mortgage debt based on interest rates, whether they adjust (and if so, where you think interest rates may go throughout its life), whether they have a balloon payment, tax deductibility, flexibility of payments, etc.

If you don’t yet have emergency savings, that should be a top priority. If you have dependents but don’t yet have life or disability insurance, those should be high priorities as well. If you have children, funding their college savings would probably be a higher priority.
But if you have no other debts and have the other important bases covered, then paying down your mortgage is probably a very good decision, depending on how much you value the peace of mind you get from knowing you cannot be evicted even if you lose your job and cannot make mortgage payments. It also depends on your interest rate and other terms, as well as what you expect to get if you invested the possible mortgage prepayments elsewhere.
Please note, this decision should not depend on whether you think the house will increase or decrease in value because you already own the house. You are not choosing whether or not to put more money into it but whether or not to more quickly reduce the debt to your mortgage issuer. As an investment choice, only the terms of the debt you are repaying are relevant when comparing mortgage prepayment versus your other investment options.
We are lucky to be able to own our home with no mortgage debt (it admittedly took both personal savings, luck, and family assistance), and we value that peace of mind very highly due to our personal experience in the financial crisis. But see where your own priorities and values lie, because there is no one correct answer to this question.
Originally posted at Quora