Mr Trump is trying to balance working with China on relations with North Korea, with his "America-first" trade views.
Beijing warned that it "will not sit idle" if the probe leads to sanctions.
Mr Trump returned to Washington to sign the order, which authorises US Trade Representative Robert Lighthizer to explore whether to undertake a deeper Section 301 investigation.
If such a probe occurs and finds against China, the president could unilaterally impose tariffs, sanctions or other trade restrictions to protect US industries.
The initial review is expected to take months.
Why is the Trump administration concerned?
Donald Trump has long railed against the massive US trade deficit with China.
The total trade relationship was worth $648bn (£500bn) last year, but trade was heavily skewed in China's favour with the US amassing a nearly $310bn deficit last year.
Some of that deficit, the argument goes, is because Chinese firms are copying US products and ideas and either selling them back to the US at a lower price or squeezing US imports out of the Chinese market.
Concern over counterfeit goods and online piracy also pre-date the Trump administration.
US firms are especially upset about rules that require local partnerships or disclosure of intellectual property to enter the Chinese market, which they say facilitates transfer of their ideas.
How much does intellectual property infringement cost?
The Commission on the Theft of American Intellectual Property estimates that the annual cost to the US economy from counterfeit goods, pirated software and theft of trade secrets is between $225bn and $600bn.
The commission says that China is the world's principal intellectual property infringer and that it accounts for 87% of counterfeit items coming into the US.
In November 2015, the Office of the Director of National Intelligence put the cost of economic espionage through hacking at $400bn a year.
The European Union, Japan, Germany and Canada have all expressed concern over China's behaviour on intellectual property theft.
What was the reaction in the US?
Mr Trump called the memo "a very big move", but also noted that "this is just the beginning".
The measure was overshadowed by controversy over his condemnation of the white supremacist groups involved in violence in Charlottesville, Virginia on Saturday, seen by many as dilatory.
Top Democrat Sen Chuck Schumer dismissed the memo as "tough talk" without action. The US already reviewed intellectual property practices in China earlier this year as part of a wider report.
"To make an announcement that they're going to decide whether to have an investigation on China's well-documented theft of our intellectual property is another signal to China that it is OK to keep stealing," Mr Schumer said in a statement.
Analysts for the Eurasia Group said the decision suggested the administration had decided to prioritise co-operation with China over North Korea.
"Rightly or wrongly, the White House believes that taking tough trade actions against China will harm its ability to elicit more co-operation from China on North Korea. This decision captures that prioritisation more than any other to date," analysts wrote in a note issued on Monday.
Unravelling trade system
Some critics say President Trump's push is a dangerous move that could cause the international trade system to unravel.
While intellectual property theft and copyright infringement are major problems, pursuing them with such a blunt instrument could prompt China to take retaliatory measures, according to Deborah Elms from the Asian Trade Centre.
"There will be a lot of collateral damage along the way. I understand the impulse to get tough, but if I were a company in China, I would be very worried about this," she says.
There were hundreds of section 301 investigations in the 1970s and 1980s, but the policy tool was largely set aside after the WTO brought into effect a binding dispute system, largely at the prompting of the US.
Ms Elms says that if the US casts aside the system it helped to develop over 30 years, other countries might be tempted to act unilaterally too.
What does China say?
In response to the move China's commerce ministry issued a statement voicing "serious concern" and warning this would "definitely harm bilateral trade relations".
"If the US side take actions that impair the mutual trade relations, disregarding the facts and disrespecting multilateral trade rules, China will not sit idle," the statement said on Tuesday.
Official media in China have criticised the measure too.
A Xinhua News Agency commentary labelled the move "outdated" and said it would hurt both countries.
In an editorial, the official China Daily urged the Trump administration to pursue a different path.
It said the move could not be seen in isolation from North Korea, and what President Trump sees as China's failure to rein in the country's nuclear ambitions and missile programme.
"Instead of advancing the United States' interests, politicising trade will only acerbate the country's economic woes, and poison the overall China-US relationship," it said.