It would be pretty hard to tell for sure. We would see stars or planets being pulled toward a seemingly empty patch of space, which would mean that there is something heavy there. By measuring how hard the stars or planets are being pulled, we could figure out how heavy is the object doing the pulling. If it's heavy enough, and small enough, we would guess that it must be a black hole. We could be much more sure if we saw the gravitational waves coming from something falling into a black hole. These waves would show us very precisely what spacetime looks like near the place they were produced. Thus, gravitational waves give us a way to really "see" a black hole.