Span text pushes other elements (buttons) to the right and left

I am trying to create an voting system for my website but am currently struggling with upvote and downvote counters styling. Here is how it looks right now:

As you can see, the problem is that whenever the number on the right or left side of the buttons gets large, it pushes other elements. I want the two buttons to stay in the center and the upvote counter to grow to the left, and downvote to the right. This way, everything will be centered.

Here is my current CSS code:

.upvoteButtonsContainer{
  display: flex;
  justify-content: center;
  margin-top: 15px;
}

.upvoteButton{
  margin-right: 2px;
  border: none;
  background-color: transparent;
  box-shadow: rgba(0, 0, 0, 0.24) 0px 3px 8px;
  border-radius: 5px;
  font-size: 25px;
}

.downvoteButton{
  margin-left: 2px;
  border: none;
  background-color: transparent;
  box-shadow: rgba(0, 0, 0, 0.24) 0px 3px 8px;
  border-radius: 5px;
  font-size: 25px;
}

.upvoteCounterContainer{
  display: block;
  direction: rtl;
  text-align: right;
}

.downvoteCounterContainer{
  display: block;
  direction: ltr;
  text-align: left;
}

.upvoteCounter{
  margin-right: 5px;
}

.downvoteCounter{
  margin-left: 5px;
}

And my HTML code:

<div className="homePage">
      {imageLists.map((image) => {
        return (
          <div className="container">
            <div className="meme">
              <img src={image} className="meme-img"></img>
            </div>
            <div className="upvoteButtonsContainer">
              <div className="upvoteCounterContainer">
                <span className="upvoteCounter">120</span>
              </div>
              <button type="button" className="upvoteButton">
                👍
              </button>
              <button type="button" className="downvoteButton">
                👎
              </button>
              <div className="downvoteCounterContainer">
                <span className="downvoteCounter">1241123123310</span>
              </div>
            </div>
          </div>
        );
      })}
    </div>

Thank you!

Answer

If you always want the buttons to be center and dont care about the numbers breaking into new lines for exceptionally large numbers, you can solve this by position: absolute quite easily:

.upvoteButton{
  position: relative;
}

.downvoteButton{
  position: relative;
}

.upvoteCounterContainer{
  position: absolute;
  top: 50%;
  transform: translateY(-50%);
  right: calc(100% + 5px);
}

.downvoteCounterContainer{
  position: absolute;
  top: 50%;
  transform: translateY(-50%);
  left: calc(100% + 5px);
}

And then wrapping the counters inside the buttons:

<div className="upvoteButtonsContainer">
  <button type="button" className="upvoteButton">
    👍
    <div className="upvoteCounterContainer">
      <span className="upvoteCounter">120</span>
    </div>
  </button>
  <button type="button" className="downvoteButton">
    👎
    <div className="downvoteCounterContainer">
      <span className="downvoteCounter">1241123123310</span>
    </div>
  </button>
</div>

Extra changes: You can remove all text align/direction styles If this doesnt work properly you can try putting the icons in specific containers (Divs/spans) If the counters dont get an auto width (their width doesnt grow based on their content) you can calculate and give them a fixed value.