Staff Software Engineer Lakehouse

At eBay were more than a global ecommerce leader were changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. Were committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass authenticity thrives bold ideas are welcome and everyone can bring their unique selves to work every day. Were in this together sustaining the future of our customers our company and our planet.

Join a team of passionate thinkers innovators and dreamers and help us connect people and build communities to create economic opportunity for all.

About the Team and the Role
We are part of the Analytics Data Platform Team within eBay working with a mission towards improving availability security performance tools and other features within Analytics Platform.

We are seeking an experienced Software Engineer with a strong background in Spark/Lakehouse table format technologies responsible to build and optimize our next-generation lakehouse platform.

What You Will Accomplish

• Our focus is on enhancing Spark stability/usability/performance and extending Icebergs capabilities.
• Additionally we are building internal storage and metadata services to power our lakehouse architecture.
• Troubleshoot and resolve production issues ensuring minimal downtime.
• Maintain and improve system monitoring logging and alerting.

What You Will Bring

• A validated successful track record of 5 years with strong programming skills in either Java or Scala.
• Comprehensive knowledge of Apache Sparks internals including Catalyst optimizer Tungsten execution engine shuffle mechanics AQE and memory management.
• Solid understanding of file formats like Apache Parquet and table formats such as Apache Iceberg Delta or Apache Hudi internals including snapshots manifests partition specs and table metadata.
• Familiarity with Linux/Unix systems including shell scripting and system commands.
• Understanding of networking principles as Hadoop often operates in distributed environments. Overall the candidate is expected to have analytical skills to tackle complex distributed system challenges and optimize solutions.
• Excellent problem solving and analytical skill on both defect and performance issue resolution.
• Great teammate and able to work with a global team.

Plus Points

• Contribution to related open source projects like Apache Spark/Iceberg.
• Experience in designing and implementing distributed systems.
• Good understanding of other Hadoop related projects including HDFS/HBase/Yarn.

The base pay range for this position is expected in the range below:

Base pay offered may vary depending on multiple individualized factors including location skills and experience. The total compensation package for this position may also include other elements including a target bonus and restricted stock units (as applicable) in addition to a full range of medical financial and/or other benefits (including 401(k) eligibility and various paid time off benefits such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

If hired employees will be in an at-will position and the Company reserves the right to modify base salary (as well as any other discretionary payment or compensation program) at any time including for reasons related to individual performance Company or individual department/team performance and market factors.

Additional Details

eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race color religion national origin sex sexual orientation gender identity veteran status and disability or other legally protected you have a need that requires accommodation please contact us at. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBays commitment to ensuring digital accessibility for people with disabilities. It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. To learn how we handle your personal data and use AI responsibly please visit ourTalent Privacy Notice Privacy Center and AI Hiring Guidelines.