Senior Big Data (Hadoop/Spark) Engineer
Oct 2, 2020
At M&T Tech, we’re a team of makers, doers, and builders, working to create the most advanced technology solutions in banking. We’re not your stereotypical suit and tie bankers: We’re an innovative team of leading tech experts, pushing boundaries, and taking risks. We’re building an agile team of the most skilled and creative, working to solve complex problems, architecting solutions, writing high-performance software, and charting our new path, all to make the lives of our customers, and the communities that we serve better. Join us and be part of something new as we build tomorrow’s bank, today.
The Enterprise Architecture, Data, and Engineering Group is responsible with enabling our technology partners across the Bank with a suite of best-in-class Platforms, Tools, and Products; allowing them to deliver rich customer solutions all while decreasing their time to market. In addition, the Group creates significant value by providing an advisory and governance function on these capabilities, so technology delivery teams can focus on the “What” not the “How”.
This position is an opportunity to build the data framework with HDFS, Spark and Kafka technology for batch and streaming analytics. Data processes in BDM, if appropriate, include ingestion, standardization, metadata management, business rule curation, data enhancement, and statistical computation against data sources that include relational, XML, JSON, streaming, REST API, and unstructured data. BDM has provided a metadata injection layer for data ingestion across six SQL Server sources that will be leveraged for additional source ingestion.
Scope of Responsibilities:
Understand, prepare, process and analyze data to drive operational, analytical and strategic business decisions. You will create, modify and maintain both Sqoop and BDM code and complex SQL for BI/DW data flows. You will program in Spark and Python where BDM Spark co-generation is not adequate. Build end to end data flows from sources to fully curated and enhanced data sets. This can include the effort to locate and analyze source data, create data flows to extract, profile, and store ingested data, define and build data cleansing and imputation, map to a common data model, transform to satisfy business rules and statistical computations, and validate data content.
Informatica tools: PowerCenter, MDM, Big Data Manager
Databases: SQL, NoSQL, Postgres, Cassandra
AWS cloud services: EC2, EMR, RDS, Redshift
Stream-processing systems: Storm, Spark-Streaming
The successful candidate will have:
Minimum of an associate degree and 8 years’ systems analysis/application development experience, or in lieu of a degree, a combined minimum of 10 years’ higher education and/or work experience
Experience with Agile Methodology
An ability to build out data products & product enhancements from idea through to launch
Strong collaboration with technology partners and customers on feature requirements and prioritization
A team player mindset with an ability to thrive and effectively communicate in a fast-paced, constantly evolving environment
M&T Bank is a Top 20 US bank holding company and one of the best performing and financial stable regional banks in the country, we offer our technology employees a wide range of performance-based career development opportunities. We have a strong commitment to our customers and the communities we serve, and we continue to grow with a focus on the future. So, when looking to advance your career, look to M&T. Grow with us.