These terms sometimes trip people up and there's no reason for it. The person who is talking implies and the person who is listening infers. Here are examples:
Imply – to insinuate. Helen implied that she was running out of money.
Infer – to speculate. From what Helen said, Dahlia inferred that Helen was running out of money.