I would suggest choosing a DSLR over a compact camera as the larger sensor size will offer you greater image quality, better ISO performance ect.
On this comment as a heads up, you may read comments here and elsewhere that the "kit" or entry-level lenses with DSLRs/SLTs give you generally poor image quality or limited performance but that statement is relative. Compared to more expensive DSLR lenses of similar focal range, there's a degree of truth in such statements but compared to most compact cameras, the kit lenses with DSLRs give are vastly superior in terms of image quality.
The other benefit that DSLRs have that's often impractical to replicate with a compact is the ease of blurring the background to make the subject/focus of the photo stand out. I'd imagine that if you plan on doing action shots of horses, you'd probably want to get a shot of the horse in mid-jump over a fence, which I reckon is a great opportunity to get that effect. A fast lens (you'll learn the terminology soon enough) will make this effect easier to achieve but such lenses are usually expensive except for the entry-level 50mm.