Tsunamis are low-probability high-consequence events, usually caused by an earthquake in the ocean and can result in high casualty rates and billions of dollars in damage. Tsunamis can be divided into two main categories: near-field and far-field tsunamis, based on the location of their origin with respect to the site of interest. To perform risk assessment of communities subjected to tsunamis, the current approach would be to use empirical data from historical events, making the data site specific. Recently, researchers have developed approaches to estimate the risk of structures subjected to far-field earthquake generated tsunamis using a simulated tsunami force on a structure numerically. However, for near-field tsunamis, ground motions caused by the earthquake will reach the structure earlier than the tsunami, damaging the structure, which can substantially impair its structural performance in the following tsunami. The multi-hazard case of tsunami following earthquake is discussed herein and a physics-based approach to estimate the risk of structures subjected to them is presented. An illustrative example is presented to elaborate the methodology for a steel building. Successive nonlinear analyses are used to develop fragility functions based on joint earthquake-tsunami intensity measures (spectral acceleration-flow depth-flow velocity). These functions are used in combination with hazard analysis of a specific location to obtain loss estimates. Three different approaches were used for this process and the results showed that the use of the joint three intensity-measure fragilities is essential for the accuracy when estimating damage or structural loss and neglecting their interaction results in substantial errors.