The phrase negative SEO seems to have two fairly distinct meanings, and to ensure I properly answer this question I’ll address both.
Negative SEO is Practices Google Prohibits
A good example of such a practice is buying links from companies that accept money in exchange for links.
What’s important here is to understand that while we sometimes attribute human capabilities to Google, Google neither reads, nor understands.
The Google search engine robot is software. It loads details about web pages into a massive database and performs large scale data analysis in an effort to infer web page quality for purposes of ranking web pages.
The best way to describe what Google does is statistical analysis and pattern analysis.
It seems reasonable to believe (but only the people at Google know for sure) that the first level of filtering is done by software.
When the link structure pointing to a page, and the keyword density within the page, etc, etc, etc falls outside understood patterns and parameters the page is suspect.
Google does hire human raters to rate the quality of web pages contained in search engine result pages and I hope (but again only the people at Google know for sure) that for web pages that are slightly outside understood patterns a person checks it out and makes a final determination.
If you’re looking for specifics of what Google feels are activities which artificially inflate the search ranking of low quality web pages, refer to the Google Search Quality Rating Guide, as they’re very clear.
Negative SEO Inflicted on a Competitor Website
While it should be fairly easy for Google to determine that negative SEO practices are being used in an effort to artificially inflate the search ranking of low quality content, I see no way for Google to determine WHO is doing the negative SEO.
If I run Company A and my director competitor is Company B can I hire a low quality link building service to build hundreds if not thousands of low quality links pointing to the website of Company B, my competitor.
Sure I could. Of course my intention would be to lower their ranking.
How would Google know they did not inflict this on themselves?
This is a tough one, and this is why we all need to be vigilant about the search rankings of our sites and the links coming into our sites.
It is theoretically possible we need to disavow bad links created by someone who intentionally wishes to lower our search ranking.
Learn More About What it Takes to PROPERLY Rank Your Site…SEO Needs Link Building