Nah, this is totally the reasonable way to do it, iff you can tolerate the compression loss or whatever. Because 4k60 is like 12Gbps uncompressed, and even more after you cram ethernet headers onto everything. So most such devices include some compression, and the really expensive ones let you configure how much.

Failing that, you're probably doing SDI over your own lambda.