You can remove unwanted activities and reports by marking them inactive.
Go to Administration -> Program Settings -> Program Activity Types or Program Report types.
Edit the activity or report type, check "Inactive", and save.
Note: Any existing activities or reports based upon this type you mark inactive will be unaffected and remain the case file tab.