Firstly, you drag the file to the trash. Then, just empty the trash by right-clicking and then selecting "Empty Trash"

The trash is similar to the recycle bin in Windows.