A fusion category is called pointed if every simple object is invertible under the monoidal product. These are described by finite groups together with U(1) valued 3-cocycles on G. We synthesize ideas from Eilenberg-MacLane, Sutherland, and Jones to describe a general scheme for building actions of pointed fusion categories with non-trivial 3-cocycles on ordinary crossed products. We use this to show that for every pointed fusion category D and every n>2, there exists a closed connected n-manifold M and an action of D on the commutative C*-algebra C(M).