If it doesn't challenge you, it doesn't change you. Anyone who trains with me will have heard this somewhere along the way ;) 'But it's hard', 'I can't do it': yes, it's hard - but just because you can't do it today, doesn't mean you won't do it tomorrow. When we don't challenge ourselves, we stop finding a sense of achievement in our activities. I truly believe that our sense of worth, our innate belief in ourselves, lies in that sense of accomplishment. We need to overcome challenges. We need to see improvement, we need to push a little further each time. Our bodies and our minds adapt to every new 'threat' very quickly and so it's important to add new challenges as we do adapt. Sure, it's easier not to try. It's way easier not to challenge yourself. Some days we can handle bigger challenges than others. In 10 years or 20 years, when you look back, if your only regret is that you didn't just try, how will you justify that to yourself?
Your only fear should be that of never realising your own potential. Recently, I met someone who made me see that our limits are never set in stone. We create them; we adhere to them. We are constantly working within a set of rules that we create for ourselves. They can be changed, altered, shifted. It takes work, effort and consistency. It takes dragging the motivation out of yourself and being clear and honest about what you want. It takes a LOT of work. But if you reset your own limits successfully, you'll be on the brink of finally believing the extent of what you can achieve.