We study the astrophysical impact of inaccurate and incomplete modeling of the gravitational waveforms from compact binary coalescences (CBCs). We do so by the matched filtering of phenomenological inspiral-merger-ringdown (IMR) signals with a bank of inspiral-phase templates modeled on the 3.5 post-Newtonian TaylorT1 approximant. The rationale for the choice of the templates is threefold. (1) The inspiral phase of the phenomenological IMR signals, which are an example of complete IMR signals, is modeled on the same TaylorT1 approximant. (2) In the low-mass limit, where the merger and ringdown phases are much shorter than the inspiral phase, the errors should tend to vanishingly small values and, thus, provide an important check on the numerical aspects of our simulations. (3) Since the binary black hole signals are not yet known for mass ratios above ten and since signals from CBCs involving neutron stars are affected by uncertainties in the knowledge of their equation of state, inspiral templates are still in use in searches for those signals. The results from our numerical simulations are compared with analytical calculations of the systematic errors using the Fisher matrix on the template parameter space. We find that the loss in signal-to-noise ratio (SNR) can be as large as 45% even for binary black holes with component masses m1 = 10 M ⊙ and m2 = 40 M⊙. Also the estimated total mass for the same pair can be off by as much as 20%. Both of these are worse for some higher mass combinations. Even the estimation of the symmetric mass ratio η suffers a nearly 20% error for this example and can be worse than 50% for the mass ranges studied here. These errors significantly dominate their statistical counterparts (at a nominal SNR of 10). It may, however, be possible to mitigate the loss in SNR by allowing for templates with unphysical values of η.