You don't have to fix absolutely everything, but we recommend you do as much as you can. Use the results we provide you as a guideline, and use your own judgment. Sometimes, an element might be technically invalid according to what our tools have access to, but it actually makes sense for you to have it as is on your website.
For example, the speedtest might say that an image is too large, but if scaling it down or optimizing it any further alters the quality to the point where it affects the design or the user experience, we suggest you leave it as is.
Another example: our tool might detect that you are using an element that isn't supported on some browsers, but if you have a shim or a polyfill that helps fix that, you should most likely ignore this error. Our tools are not set up to detect use of shims and polyfills at the moment.