Removing the Delete permission from the Shipper profile and using a permission set to remove the Delete permission are both valid ways to prevent users from deleting delivery records. Removing the delete button from the Delivery page layout will not prevent users from deleting records using other methods, such as list views or reports. Changing the organization-wide default of deliveries to Private will not affect the delete permission, but only the sharing settings.