Just discovered that neovim supports highlight the line number, and gitgutter utilizes this. Not sure I'll keep this, but I thought it looked pretty good