Was the person you are looking for born in Morayshire or Elginshire as it was known, but you can't find them there now??
Try looking at the 'strays' lists. The main listing lists people who, in 1881, on census night not were not in Morayshire for whatever reason, either visiting, working away or those who had permenantly moved out of the county.
The secondary list is for people who have moved out of the county and have been found in other census years or other documentation
Strays found in other census and /or documented sources