New versions usually mean fewer bugs and more features, and this is exactly the case.
Version 4 has fixed 10 bugs and added more than 10 enhancements and new features. You can check this out in the fullPage.js changelog for version 4.
Not only that.
fullPage.js version 3 will no longer receive any updates and the support for it will be limited to answering questions of usage.
Is up to you to decide if you can still run your pages using version 3 or version 4.
You might also find these articles relevant: