phase response of an analog filter

A non-linear phase response is equivalent to group delay variation, and similar to multi-path fading can cause inter-symbol interference if it is severe enough, but either can be corrected with channel equalization. In this case it would be a static correction so would not be overly complicated to implement in the transmitter, receiver or both. Factoring the equalization into the cascade of two equal filters and then implementing it in both the transmitter and receiver would also provide the benefit of SNR optimization (against white noise) as a matched filter (but the actual performance benefit of doing that may be negligible compared to the added complication, so just throwing out the thought).

The metric to analyze would be group delay variation since that would show it directly. A linear phase component is only a delay and does not provide any distortion, so the group delay which is the derivative of the phase with respect to frequency would convert any linear phase to a constant value and highlight the variation which is the destructive component. One approach is to simulate the filter response and analyze the eye diagram to assess the signal quality and if further equalization and or changes to the analog filter design would be needed.

The graphic below depicts this for the case of multipath, but multipath is simply a filter with phase (and amplitude) variation across the signal bandwidth.