Using Graphics.DrawMeshInstancedIndirect so that I can calculate the fish positions with compute shader.
The moving tails are just vertex displacements in shader. Rotation are also done in the rendering shader.
Below is the look-at matrix that takes the normalized velocity to be the rotation, and to be the forward axis.
float4 ApplyRotation (float4 v, float3 rotation)
{
// Create LookAt matrix
float3 up = float3(0,1,0);
float3 zaxis = rotation;
float3 xaxis = normalize(cross(up, zaxis));
float3 yaxis = cross(zaxis, xaxis);
float4x4 lookatMatrix = {
xaxis.x, yaxis.x, zaxis.x, 0,
xaxis.y, yaxis.y, zaxis.y, 0,
xaxis.z, yaxis.z, zaxis.z, 0,
0, 0, 0, 1
};
return mul(lookatMatrix,v);
}

Next, I used the shader from here to make the fishes reacts to light. Put some post-processing effects and fog to make it looks a bit better. One point to note, the normal has to rotate the same way that the vertices do.