Feed
A feed, or a web-feed, is a special file that contains content updates. It allows anyone to subscribe to these updates, providing they have a feed reader. Whenever a new update happens, subscribers will see it appear on their feed reader. This makes feeds incredibly useful: they allow us to aggregate all the content we like without having to manually check for updates. Since it's platform agnostic, it's in my opinion a much saner alternative to social medias' "follow" feature. If you don't have a feed yet, I'd highly suggest creating one!
When reading about feeds, the word "syndication" is often used. It's an analogy borrowed from print/radio media, where a "syndicate" owns the rights to a content, and can "syndicate" (sell) that content to other newspapers or radios. On the web, content distributors are said to "syndicate" a feed, meaning to allow users to subscribe to it.
If you're looking to know more about my views on feeds, check this post.
RSS
RSS is as of today the most well known feed format. What's interesting about it is its history. I'd highly suggest reading it if you are into that. There are two different projects that borrow the same initials: RDF Site Summary (RSS) 1 and Really Simple Syndication (RSS) 2.
Note that at times, the term "RSS" and "feed" is used interchangeably.
Atom
Because of the complicated history of RSS, and in order to remove ambiguity to the specification, a third project exists: atom. I personally enjoy Atom the most, so that's the format I use all the time.
Backlinks:
software i like status cafe blog html journal subscribing to a journal page writing atom feed manually journal html journal and html blog formats bookmarklet subscribing to a blog microformat html blog opml