A new set of guidelines from the Department of Housing and Urban Development will make it tougher for landlords and home sellers to discriminate against applicants who have criminal backgrounds.

The guidance, announced Monday by HUD, means that a flat-out refusal to rent or sell to people who have criminal records is discriminatory because minorities—African Americans and Latinos in particular—are disproportionately arrested and imprisoned.

"No American should ever be discriminated against because of their race or ethnicity, even if that discrimination results from a policy that appears neutral on its face," HUD Secretary Julián Castro said Monday during the National Low Income Housing Coalition Policy Forum in Washington D.C. "Black and Latino Americans are unfairly arrested at significantly higher rates than white Americans."

Under the new guidelines landlords will have to better scrutinize whether a person was arrested and if they were also convicted. And, even if a person was convicted, property owners have to weigh the nature and severity of the crime and conviction when considering an applicant's housing application.

Failure to do so might mean the homeowner could potentially face an investigation for discrimination and, ultimately, civil penalties.