General settings

1. Go to Google Calendar website
2. Open settings by clicking the gear
3. Event Settings > Automatically add invitations > No, only show invitations to which I've responded
4. View Options > uncheck Show declined events
5. Events from Gmail > uncheck Automatically add events from Gmail to my calendar

App settings

1. Open the Google Calendar app
2. Menu > Settings > General > uncheck Show declined events

Source: Lifehacker.com